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LIST OF ACRONYMS 


BAt+ 
BHO 
CAHPS 


CDC 


CHI-SQUARE 


Cl 


CMHC 


CMS 


CPT 


DHHS 





Blue-Advantage Plus of Kansas City 
Behavioral Health Management Organization 
Consumer Assessment of Health Plans Survey 
Centers for Disease Control and Prevention 


A statistical test that is used to examine the probability of a change or 
difference in rates is due to chance. 


Confidence Interval 
Community Mental Health Center 


Centers for Medicare and Medicaid Services, U.S. Department of Health 
and Human Services 


Current Procedural Terminology 

Calendar Year 

U.S. Department of Health and Human Services 
Missouri Department of Health and Senior Services 
Missouri Department of Social Services 

Early, Periodic Screening, Diagnosis and Treatment 
External Quality Review 

External Quality Review Organization 

MO HealthNet Fee-for-Service 

Healthcare USA 


MO HealthNet Healthy Children and Youth, the Missouri Medicaid 
EPSDT program 


Healthcare Effectiveness Data and Information Set 
Health Insurance Portability and Accountability Act 


Health Information Systems 
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HMO 


HOME STATE 


ICD-9 


ICN 


ISCA 


LPHA 


MBE 


MC+ 


MC+ MCOs 


MCHP 


MCO 


MDIFP 


MMIS 


MO HEALTHNET 


MO HEALTHNET 
MCHPs 
MO CARE 


MOHSAIC 


NCPDP 


NCQA 





Section | 
Preparation for the EQR 
Health Maintenance Organization 
Home State Health Plan of Missouri 


International Classification of Diseases, Ninth Revision, Clinical 
Modification, World Health Organization 


Internal Control Number 

Information Systems Capability Assessment 
Local Public Health Agency 
Minority-owned Business Enterprise 


The name of the Missouri Medicaid Program for families, children, and 
pregnant women, prior to July 2007. 


Missouri Medicaid Program Managed Care Organizations (prior to July 
2007) 


Managed Care Health Plan 
Managed Care Organization 


Missouri Department of Insurance, Financial Institutions and Professional 
Registration 


Medicaid Management Information System 


The name of the Missouri Medicaid Program for families, children, and 
pregnant women. 


Missouri Medicaid Program Managed Care Health Plans 


Missouri Care Health Plan 


Missouri Public Health Integrated Information System 


National Council for Prescription Drug Program 


National Committee for Quality Assurance 
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N.S Not significant, indicating that a statistical test does not result in the ability 
= to conclude that a real effect exists. 


NSE/CMS 1500 National Standard Format/ Center for Medicare and Medicaid Services Form 


1500 
PCP Primary Care Provider 
PIHP Prepaid Inpatient Health Plan 
PIP Performance Improvement Project 
PRO Peer Review Organization 


MO HealthNet Managed Care Quality Assessment and Improvement 


Cae) Advisory Group 


QI/UM 


; Quality Improvement/Utilization Management Coordinator 
Coordinator 


State Medicaid Agency, the Missouri Department of Social Services, MO 


ann HealthNet Division 
State Public Health Agency, the Missouri Department of Health and Senior 
SPHA 
Services 
UB-92 Universal Billing Form 92 
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Administrative Method The Administrative Method of calculating HEDIS Performance Measures 


Confidence interval or 


level 


Hybrid Method 


Interrater reliability 


(IRR) 


Probability sample 


Random sample 


requires the MCHP to identify the denominator and numerator using 
transaction data or other administrative databases. The Administrative 
Method outlines the collection and calculation of a measure using only 
administrative data, including a description of the denominator (i.e., the 
entire eligible population), the numerator requirements (i.e., the 
indicated treatment or procedure) and any exclusion(s) allowed for the 


measure. 


The range of accuracy of a population estimate obtained from a sample. 


Hybrid Method requires the MCHP to identify the numerator through 
both administrative and medical record data. The MCHP reports a rate 
based on members in the sample who are found through either 
administrative or medical record data to have received the service 


identified in the numerator. 


A method of addressing the internal validity of a study by ensuring that 


data are collected in a consistent manner across data collectors. 


A sample in which every element in the sampling frame has a known, 
non-zero probability of being included in a sample. This produces 
unbiased estimates of population parameters that are linear functions of 


the observations from the sample data!. 


Selection of sampling units from a sampling frame where each unit has 


an equal probability of selection. 





‘Levy, P.S., Lemeshow, S. (1999). Sampling of Populations: Methods and Applications, Third Edition. John 


Wiley and Sons: New York. 
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Reliability The consistency of findings across time, situations, or raters. 
Sampling frame The population of potential sampling units that meet the criteria for 


selection (e.g., Medical encounter claim types from January |, 2004 


through March 31, 2004). 


Sampling unit Each unit in the sampling frame (e.g., an encounter). 
Simple sample Selection of sampling units from one sampling frame. 
Unpaid claim All unpaid and denied claims from the MCHP,; All claims not paid by the 


MCHP either through capitation or through other payment 


methodology. 
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PREPARATION WITH THE STATE MEDICAID AGENCY 

Effective February |, 2014 the State of Missouri contract for the External Quality Review of the MO 
HealthNet Managed Care Program (State of Missouri Contract No: C312155001, Amendment No.: 
003) was awarded to comply with federal requirements for states to contract with an external, 
independent entity to implement the mandatory protocols for External Quality Review. Monthly 
meetings for planning the scope of work, technical methods and objectives, are scheduled beginning 
each January for the upcoming review year. Monthly meetings are held with the SMA and the 
EQRO throughout the review period. Additional meetings and teleconference calls may be 


conducted as needed between SMA and EQRO personnel. 


At the first meeting of each year, the previous years’ report is discussed and the plan for the 
subsequent audit is initiated. The EQRO clarifies the SMA’s objectives for each of the protocols, 
develops data requests, prepares detailed proposals for the implementation and analysis of data for 
each protocol, and prepares materials for SMA review. Plans are made to conduct Orientation 
Conference Calls for the upcoming EQR with each MO HealthNet Managed Care Health Plan 
(MCHP) that are attended by the SMA. Written proposals for each protocol are developed and 


approved by the SMA indicating differences in the approach or information to be validated. 


PREPARATION OF MCHPS 

To prepare the MCHP for the implementation of the yearly EQR an annual Orientation Conference 
Call is conducted by the EQRO Project Director and personnel. The EQRO Project Director and 
personnel conduct orientation to the protocols and the EQR processes with each MCHP. In 
addition, the EQRO Project Director presents a timeline for project implementation and answers 
MCHP questions at a combined MO HealthNet Managed Care QA&l Advisory Group/MO 
HealthNet Managed Care All-Plan meeting. 


The EQRO Assistant Project Director arranges the dates of the teleconference calls with MCHP 
QI/UM Coordinators or Plan Administrators. A detailed presentation, tentative list of data 
requests, and the proposals approved by the SMA are sent to MCHPs prior to the teleconference 
orientation sessions. MCHPs are requested to have all personnel involved in fulfilling the requests 
or in implementing activities related to the protocols (e.g., performance improvement projects to be 
validated, performance measures to be validated) present at the teleconference calls. The 
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orientation presentation is contained in Appendix |. An SMA representative is invited to attend all 
conference calls. Notes are sent regarding any calls the SMA does not attend. To avoid confusion 
and the inundation of multiple requests at once, the requests for information from MCHPs are 
normally implemented in a staged approach from January through April. All communications 
(letters, general and specific instructions) are approved by the SMA prior to sending them to the 


MCHPs. 


DEVELOPMENT OF WORKSHEETS, TOOLS, AND RATING CRITERIA 

The EQRO Project Director, Assistant Project Director, and a healthcare consultant are responsible 
for modifying the worksheets and tools used by the EQRO during each audit. The EQRO Assistant 
Project Director revises the worksheet (Attachment B) for Validating Performance Improvement 


Project Protocol to add details specific to the MO HealthNet Managed Care Program each year. 


The Validating Performance Measures Protocol worksheets are revised and updated by the EQRO 
Project Director to reflect the Performance Measures selected for review for the appropriate 
HEDIS year. The worksheets were developed by Behavioral Health Concepts Inc. staff are updated 


annually to reflect the information needed for that year’s audit. 


The SMA continues to conduct the activities of the MO HealthNet Managed Care Compliance with 
Managed Care Regulations Protocol through the state contract compliance monitoring process. 
The work of the EQRO involves the review and evaluation of this information (see Medicaid 
Program; External Quality Review of Medicaid Managed Care Organizations of 2003, CFR §438.58). 
The state contract for EQRO requires the review of SMA’s activities with regard to the Protocol. 
Additional policies and documents are requested prior to and during the on-site visits with MCHPs 
when information was incomplete or unclear. To facilitate the review of compliance with federal 
regulations, the EQRO Assistant Project Director works with SMA staff to develop the focus of 
each year’s compliance review to ensure that it addresses issues of concern where compliance may 
be compromised. Focused interview tools are developed and submitted to the SMA for review and 
approval. The MO HealthNet Managed Care Program consultant, who participates as part of the 


EQRO team each year reviews and assists in refinement of compliance activities. 


The EQRO utilizes the rating system developed during the 2004 audit to provide ratings for each 


MCHPs’ compliance. The SMA provides information on MCHP policy compliance with state 
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contract requirements annually. The EQRO determines if this meets the policy requirements of the 
federal regulations. The EQRO staff and the consultant review all available materials and meet with 
SMA staff to clarify SMA comments and compliance ratings. Issues are identified for follow-up at site 
visits. Updates on MCHP compliance are accepted up until the time of the on-site reviews to 
ensure that the EQRO has up-to-date information. Recommended ratings, based upon the 


preapproved rating scale are provided to SMA. 


REVIEWERS 

Three reviewers are utilized to complete all sections of the EQR. Interviews, document review, and 
data analysis activities for the Validating Performance Measure Protocol were performed by two 
reviewers from the External Quality Review Organization (EQRO). The Project Director 
conducted interviews, document review, and data analysis; she is a licensed attorney with a graduate 
degree in Health Care Administration, as well as thirteen years experience in public health and 


managed care in three states. This is her eighth External Quality Review. 


Two reviewers take primary responsibility for conducting the Performance Improvement Project 
(PIP) Validation and the Compliance Protocol activities, including interviews and document review. 
The External Quality Review Organization (EQRO) Project Director conducts backup activities, 
including assistance during the interview process, and oversight of the PIP and Compliance Protocol 
team. All reviewers are familiar with the federal regulations and the manner in which these were 
operationalized by the MO HealthNet Managed Care Program prior to the implementation of the 


protocols. 


The following sections summarize the aggregate findings and conclusions for each of the mandatory 
protocols. The full report is organized according to each protocol and contains detailed 
descriptions of the findings and conclusions (strengths, areas for improvement, and 
recommendations). In addition, it provides MCHP to MCHP comparisons and MCHP summaries for 


each protocol. 
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‘TECHNICAL METHODS 


There are three evaluation activities specified in the protocol for Validating Performance 





Improvement Projects. “Activity One: Assessing the MCOs/PIHPs Methodology for Conducting 





the PIP” consists of ten steps: 


Activity One: Assessing the MCOs /PIHPs Methodology for 
Conducting the PIP 


Step One: Review the selected study topic(s) 

Step Two: Review the study question(s) 

Step Three: Review selected study indicator(s) 

Step Four: Review the identified study population 

Step Five: Review sampling methods (if sampling was used) 
Step Six: Review the data collection procedures 

Step Seven: Assess the MCOs improvement strategies 


Step Eight: Review the data analysis and interpretation of study results 


Sf NS ws & 


IP 


Step Nine: Assess the likelihood that reported improvement is “rea 
improvement 


10. Step Ten: Assess the sustainability of documented improvement 





“Activity Two: Verifying PIP Study Findings” is optional, and involves auditing PIP data. “Activity 
Three: Evaluate Overall Reliability and Validity of Study Findings” involves accessing whether the 
results and conclusions drawn from the PIPs are valid and reliable. Activities One and Three are 


conducted by the EQRO. 


TIME FRAME AND SELECTION 

Two projects that were underway during the preceding |2 months at each MCHP are selected for 
validation. The projects to be validated are reviewed with SMA and EQRO staff after topic 
submission is complete. The intent is to identify projects which are mature enough for validation 
(i-e., planned and in the initial stages of implementation), underway or completed during the 
previous calendar year. The SMA makes the final decision regarding the actual PIPs to be validated 


from the descriptions submitted by the MCHPs. 
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PROCEDURES FOR DATA COLLECTION 
The evaluation involves review of all materials submitted by the MCHPs including, but not limited to, 
the materials listed below. During the training teleconferences MCHPs are encouraged to review 


Attachment A of the Validating Performance Improvement Projects Protocol, to ensure that they 





include supporting documents, tools, and other information necessary to evaluate the projects 
submitted, based on this tool. 


= Narrative descriptions 

"Problem identification 

= Hypotheses 

= Study questions 

" Description of interventions(s) 

= Methods of sampling 

= Planned analysis 

= Sample tools, measures, survey, etc. 

= Baseline data source and data 
"Cover letter with clarifying information 
"Overall analysis of the validity and reliability of each study 
= Evaluation of the results of the PIPs 


The EQRO Project Director, Assistant Project Director, and Review Consultant meet with the 
MCHP staff responsible for planning, conducting, and interpreting the findings of the PIPs during the 
on-site reviews occurring annually. The review focuses on the findings of projects conducted. 
MCHPs are instructed that additional information and data, not available at the time of the original 
submission, can be provided at the on-site review or shortly thereafter. The time scheduled during 
the on-site review is utilized to conduct follow-up questions, to review data obtained, and to 
provide technical assistance to MCHPs regarding the planning, implementation and credibility of 
findings from PIPs. In addition, individual clarifying questions are used to gather more information 
regarding the PIPs during the on-site interviews. The following questions were formulated and 
answered in the original documentation, or are posed to the MCHPs during the on-site review: 


«= Who was the project leader? 

= _ How was the topic identified? 

= How was the study question determined? 

«= What were the findings? 

«= What were the interventions(s)? 

«= What was the time period of the study? 

« Was the intervention effective? 

«= What did the MCHP want to learn from the study? 


All PIPs are evaluated by the Review Consultant and the Assistant Project Director. In addition, the 
projects are reviewed with follow-up suggestions posed by the Project Director, who approves final 
ratings based on all information available to the team. 
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ANALYSIS 


Criteria for identification of a PIP as outlined in the CMS protocols include the following: 
e PIPs need to have a pre-test, intervention, and post-test. 
e PIPs need to control for extraneous factors. 
e PIPs need to include an entire population. 
e Pilot projects do not constitute a PIP. 
e Satisfaction studies alone do not constitute a PIP. 
e Focused studies are not PIPs: A focused study is designed to assess processes and 
outcomes on one-time basis, while the goal of a PIP is to improve processes and outcomes 


of care over time. 


The Managed Care contract describes the following requirements for MCHP’s relative to 
conducting PIPs: 


Performance Improvement Projects: The MCHP shall conduct performance 
improvement projects that are designed to achieve, through ongoing measurements 
and intervention, significant improvement, sustained over time, in clinical care and 
nonclinical care areas that are expected to have a favorable effect on health 
outcomes and member satisfaction. As requested, the MCHP shall report the status 
and results of each performance improvement project to the state agency, which 
must include state and/or MCHP designated performance improvement projects... 
The performance improvement projects must involve the following: 


e Measurement of performance using objective quality indicators. 

e Implementation of system interventions to achieve improvement in quality. 

e Evaluation of the effectiveness of the interventions. 

e Planning and initiation of activities for increasing or sustaining improvement. 

e Completion of the performance improvement project in a reasonable time 
period so as to generally allow information on the success of performance 
improvement projects in the aggregate to produce new information on quality 
of care every year. 

e Performance measures and topics for performance improvement projects 
specified by CMS in consultation with the state agency and other stakeholders. 
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All PIPs submitted by MCHPs prior to the site visits are reviewed using an expanded version of the 
checklist for conducting Activity One, Steps | through 10, and Activity Three (Judgment of the 


Validity and Reliability of the PIPs) of the Validating Performance Improvement Projects Protocol, 





Attachment A. Because certain criteria may not be applicable for projects that are underway at the 
time of the review, some specific items may be considered as “Not Applicable.” Criteria are rated 
as “Met” if the item was applicable to the PIP, if documentation is available that addresses the item, 
and if the item could be deemed Met based on the study design. The proportion of items rated as 
“Met” is compared to the total number of items applicable for the particular PIP. Given that some 
PIPS may be underway in the first year of implementation, it is not possible to judge or interpret 
results; validity of improvement; or sustained improvements (Steps 8-10) in all instances. The final 
evaluation of the validity and reliability of studies is based on the potential for the studies to produce 
credible findings. Detailed recommendations and suggestions for improvement are made for each 
item where appropriate, and are presented in the individual MCHP summaries. Some items are 
rated as “Met” but continue to include suggestions and recommendations as a method of improving 
the information presented. The following are the general definitions of the ratings developed for 


evaluating the PIPs. 


Met: Credible, reliable, and valid methods for the item were documented. 


Partially Met : Credible, reliable, or valid methods were implied or able to be established 
for part of the item. 


Not Met: The study did not provide enough documentation to determine whether 
credible, reliable, and valid methods were employed; errors in logic were 


noted; or contradictory information was presented or interpreted 
erroneously. 


Not Applicable: Only to be used in Step 5, when there is clear indication that the entire 
population was included in the study and no sampling was conducted; or 
in Steps 8 through 10 when the study period was underway for the first 
year. 
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3.0 Performance Measures 
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3.1 Technical Methods 


Reliable and valid calculation of performance measures is a critical component to the EQRO audit. 
These calculations are necessary to calculate statewide rates, compare the performance of MCHPs 
with other MCHPs, and to compare State and MCHP performance with national benchmarked data 
for Medicaid Managed Care and/or Commercial Managed Care Organization members. These types 
of comparisons allow for better evaluation of program effectiveness and access to care. The EQRO 
reviews the selected data to assess adherence to State of Missouri requirements for MCHP 
performance measurement and reporting. The Missouri Code of State Regulations (19 CSR §10- 
5.010 Monitoring Health Maintenance Organizations) contains provisions requiring all Health 
Maintenance Organizations (HMOs) operating in the State of Missouri to submit to the SPHA 
member satisfaction survey findings and quality indicator data in formats conforming to the National 
Committee for Quality Assurance (NCQA) Health Employer Data Information Set (HEDIS) Data 
Submission Tool (DST) and all other HEDIS Technical Specifications? for performance measure 
descriptions and calculations. The State of Missouri contract for MO HealthNet Managed Care 
(C306122001, Revised Attachment 6, Quality Improvement Strategy) further stipulates that MO 
HealthNet MCHPs will follow the instructions of the SPHA for submission of HEDIS measures. 
Three measures are selected by the SMA for validation annually. These measures are required to be 
calculated and reported by MCHPs to both the SMA and the SPHA for MO HealthNet Managed 
Care Members. A review is conducted for each of the three measures selected based upon the 


HEDIS Technical Specifications. These specifications are provided in the following tables: 
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HEDIS 2013 CHILDHOOD IMMUNIZATIONS STATUS, COMBINATION 3 (CIS3) 


Description: 

The percentage of children 2 years of age who had four diphtheria, tetanus and acellular pertussis 
(DTaP); three polio (IPV); one measles, mumps and rubella (MMR); three H influenza type B (HiB); 
three hepatitis B (HepB), one chicken pox (VZV); four pneumococcal conjugate (PCV); two hepatitis 
A (HepA); two or three rotavirus (RV); and two influenza (flu) vaccines by their second birthday. 


The measure calculates a rate for each vaccine and nine separate combination rates. 


Table | - HEDIS 2013 Technical Specifications for Childhood Immunization Status (CIS) 
I. Eligible Population 


Ages Children who turn 2 years of age during the measurement year. 

Continuous 12 months prior to the child’s second birthday. 

enrollment 

Allowable gap No more than one gap in enrollment of up to 45 days during the 12 months prior 


to the child’s second birthday. To determine continuous enrollment for a 
Medicaid beneficiary for whom enrollment is verified monthly, the member may 
not have more than a 1-month gap in coverage (i.e., a member whose coverage 
lapses for 2 months [60 days] is not continuously enrolled). 


Anchor date Enrolled on the child’s second birthday.. 
Benefit Medical. 
Event/diagnosis None. 


II. Administrative Specification 





Denominator The eligible population. 

Numerators For MMR, hepatitis B, VZV and hepatitis A, count any of the following: 
e Evidence of the antigen or combination vaccine, or 
e Documented history of the illness, or 
e A seropositive test result for each antigen 


For DTaP, IPV, HiB, pneumococcal conjugate, rotavirus and influenza, count 
only: 


e Evidence of the antigen or combination vaccine. 


For combination vaccinations that require more than one antigen (i.e., DTaP and 
MMR), the organization must find evidence of all the antigens 
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DTaP 

At least four DTaP vaccinations, with different dates of service on or before the 
child’s second birthday. Do not count a vaccination administered prior to 42 days 
after birth. 


IPV 

At least three IPV vaccinations, with different dates of service on or before the 
child’s second birthday. IPV administered prior to 42 days after birth cannot be 
counted. 


MMR 
At least one MMR vaccination, with a date of service falling on or before the 
child’s second birthday. 


HiB 

At least three HiB vaccinations, with different dates of service on or before the 
child’s second birthday. HiB administered prior to 42 days after birth cannot be 
counted. 


Hepatitis B 
At least three hepatitis B vaccinations, with different dates of service on or 
before the child’s second birthday. 


VZV 
At least one VZV vaccination, with a date of service falling on or before the 
child’s second birthday. 


Pneumococcal conjugate 

At least four pneumococcal conjugate vaccinations, with different dates of 
service on or before the child’s second birthday. Do not count a vaccination 
administered prior to 42 days after birth. 


Hepatitis A 
Two hepatitis A vaccinations, with different dates of service on or before the 
child’s second birthday. 


Rotavirus 

The child must receive the required number of rotavirus vaccinations on different 
dates of service on or before the second birthday. Do not count a vaccination 
administered prior to 42 days after birth. The following vaccine combinations are 
compliant: 


e Two doses of the two-dose vaccine, or 


e One dose of the two-dose vaccine and two doses of the three-dose 
vaccine, or 


e Threedoses of the three-dose vaccine. 
The vaccines are identified by different CPT codes (Table CIS-A). 


Influenza 

Two influenza vaccinations, with different dates of service on or before the 
child’s second birthday. Do not count a vaccination administered prior to six 
months (180 days) after birth. 


Combination rates 
Calculate the following rates for Combination 2—Combination 10. 
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Combination Vaccinations for Childhood Immunization Status 


























Combination DTaP IPV MMR HiB | HepB | VZV PCV HepA _ RV __ Influenza 
Combination 2 x Xx xX x x x 

Combination 3 x x x x x x x 

Combination 4 Xx x x x xX x x x 

Combination 5 x x x x x x x x 

Combination 6 X xX xX Xx xX x x x 
Combination 7 Xx x Xx x xX x x x 

Combination 8 x x x x X x x 

Combination 9 Xx x Xx x x Xx x x 
Combination 10 X xX xX Xx Xx x x x 









































Table CIS-A: Codes to Identify Childhood Immunizations 











HWE ene Bee 
DTaP 90698, 90700, 90721, 90723 99.39 
IPV 90698, 90713, 90723 99.41 
MMR 90707, 90710 99.48 





Measles and rubella 90708 









































Measles 90705 055 99.45 
Mumps 90704 072 99.46 
Rubella 90706 056 99.47 
HiB 90645-90648, 90698, 90721, 
90748 

Hepatitis B o7a7, 074s 60010 | Vang 
VZV 90710, 90716 052, 053 
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pe netimoeeecel 90669, 90670 Go0009 
conjugate 
Hepatitis A 90663 070.0, 070.1 
Rotavirus (two dose 90681 
schedule) 
Rotavirus (three dose 
schedule) EEL 
cas 90655, 90657, 90661, 90662 | G0008 99.52 

















* ICD-9-CM Diagnosis codes indicate evidence of disease. 
** The two-dose hepatitis B antigen Recombivax is recommended for children between 11 
and 14 years of age only and is not included in this table 


Exclusion (optional) 

Children who had a contraindication for a specific vaccine may be excluded from the denominator 
for all antigen rates and the combination rates. The denominator for all rates must be the same. An 
organization that excludes contraindicated children may do so only if the administrative data do not 
indicate that the contraindicated immunization was rendered. The exclusion must have occurred by 
the second birthday. Organizations should look for exclusions as far back as possible in the 


member’s history and use the codes in Table CIS-B to identify allowable exclusions. 
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Table CIS-B: Codes to Identify Exclusions 


Immunization Description IDC-9-CM Diagnosis 





Anaphylactic reaction to the vaccine or its 


999.4 
components 


Any particular vaccine 





323.51 with (E948.4 or 
DTaP Encephalopathy £948.5 or aa 





Progressive neurologic disorder, including 
infantile spasm, uncontrolled epilepsy 





Anaphylactic reaction to streptomycin, 


IPV : ‘ 
polymyxin B or neomycin 





Immunodeficiency, including genetic 279 


PLUS MZEE CIMIM ES ae (congenital) immuno-deficiency syndromes 














HIV disease; asymptomatic HIV 042, V08 
Cancer of lymphoreticular or histiocytic 200-202 
tissue 

Multiple myeloma 203 
Leukemia 204-208 





Anaphylactic reaction to neomycin 





Anaphylactic reaction to common baker's 


Hepatitis B yeast 

















III. Hybrid Specification 


Denominator A systematic sample drawn from the eligible population for each product line. 
The organization may reduce the sample size using the current year’s 
administrative rate for the lowest rate or the prior year’s audited, product line- 
specific results for the lowest rate. Refer to the Guidelines for Calculations 
and Sampling for information on reducing sample size. 


Numerators For MMR, hepatitis B, VZV and hepatitis A, count any of the following. 


Evidence of the antigen or combination vaccine, or 


Documented history of the illness, or 
A seropositive test result 
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For DTaP, HiB, IPV, pneumococcal conjugate, rotavirus and influenza, count 
only: 
Evidence of the antigen or combination vaccine. 


For combination vaccinations that require more than one antigen (i.e., DTaP 
and MMR), the organization must find evidence of all the antigens 


Administrative Refer to Administrative Specification to identify positive numerator hits from 
the administrative data. 

For immunization evidence obtained from the medical record, the 
organization may count members where there is evidence that the antigen 
was rendered from one of the following. 

A note indicating the name of the specific antigen and the date of the 
immunization, or 

A certificate of immunization prepared by an authorized health care provider 
or agency including the specific dates and types of immunizations 
administered. 

For documented history of illness or a seropositive test result, the 
organization must find a note indicating the date of the event, which must 
have occurred by the member's second birthday. 

Notes in the medical record indicating that the member received the 
immunization “at delivery” or “in the hospital” may be counted toward the 
numerator. This applies only to immunizations that do not have minimum age 
restrictions (e.g., before 42 days after birth). A note that the “member is up to 
date” with all immunizations but which does not list the dates of all 
immunizations and the names of the immunization agents does not constitute 
sufficient evidence of immunization for HEDIS reporting. 

Immunizations documented using a generic header or “DTaP/DTP/DT” can 
be counted as evidence of DTaP. The burden on organizations to 
substantiate the DTaP antigen is excessive compared to a risk associated 
with data integrity. 


Medical record 


e For rotavirus, if documentation does not indicate whether the two-dose 
schedule or three-dose schedule was used, assume a three-dose 
schedule and find evidence that three doses were administered. 


Exclusion 


: Refer to Administrative Specification for exclusion criteria. The exclusion 
(Optional) 


must have occurred by the member’s second birthday 


Note 
This measure follows the CDC and ACIP guidelines for immunizations. HEDIS implements changes 
to the guidelines (e.g., new vaccine recommendations) after three years, to account for the 


measure’s look-back period and to allow the industry time to adapt to new guidelines 
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HEDIS 2013 FOLLOW-UP AFTER HOSPITALIZATION FOR MENTAL ILLNESS (FUH) 


The following is the definition of the Follow-Up After Hospitalization for Mental Illness measure, an 
Effectiveness of Care measure, and the specific parameters as defined by the NCQA. 

The percentage of discharges for members 6 years of age and older who were hospitalized 

for treatment of selected mental health disorders and who were seen on an outpatient 


basis or were in intermediate treatment with a mental health provider. 


Table 2 - HEDIS 2013 Technical Specifications for Follow-Up After Hospitalization for Mental Illness (FUH) 


Product lines 
Ages 


Continuous 
enrollment 


Allowable gap 
Anchor date 
Benefits 


Event/ diagnosis 


Multiple 
discharges 


Mental health 
readmission or 
direct transfer 


Non-mental 
health 
readmission or 
direct transfer 
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I. Eligible Population 


Commercial, Medicaid, Medicare (report each product line separately). 
6 years and older as of the date of discharge. 


Date of discharge through 30 days after discharge. 


No gaps in enrollment. 
None. 
Medical and mental health (inpatient and outpatient). 


Discharged from an inpatient setting of an acute care facility (including acute care 
psychiatric facilities) with a discharge date occurring on or before December 1 of the 
measurement year and a principal ICD-9-CM Diagnosis code indicating a mental health 
disorder specified below: 


295-299, 300.3, 300.4, 301, 308, 309, 311-314, 426, 430 


The MCO should not count discharges from nonacute care facilities (e.g., residential 
care or rehabilitation stays). 


A member with more than one discharge on or before December 1 of the measurement 
year with a principal diagnosis of a mental health disorder (Table FUH-A) could be 
counted more than once in the eligible population. 


If the discharge for a selected mental health disorder is followed by readmission or 
direct transfer to an acute facility for any mental health principal diagnosis within the 
30-day follow-up period, count only the readmission discharge or the discharge from 
the facility to which the member was transferred. 


Although rehospitalization might not be for a selected mental health disorder, it is 
probably for a related condition. Only readmissions with a discharge date that occurs on 
or before December 1 of the measurement year are included in the measure. Refer to 
the ICD-9-CM codes listed in Table MIP-A. 


Exclude discharges followed by readmission or direct transfer to a nonacute facility for 
any mental health principal diagnosis within the 30-day follow-up period. These 
discharges are excluded from the measure because readmission or transfer may 
prevent an outpatient follow-up visit from taking place. (Refer to Table NON-A for codes 
to identify nonacute care.) 


Exclude discharges in which the patient was transferred directly or readmitted within 30 
days after discharge to an acute or nonacute facility for a non-mental health principal 
diagnosis. These discharges are excluded from the measure because rehospitalization or 
transfer may prevent an outpatient follow-up visit. 
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Denied claims Denials of inpatient care (e.g., those resulting from members failing to get proper 
authorization) are not excluded from the measure. 


II. Administrative Specification 





Denominator The eligible population. 


Note: The eligible population for this measure is based on discharges, not 
members. It is possible for the denominator for this measure to contain 
multiple discharge records for the same individual. 


Numerators An outpatient mental health encounter or intermediate treatment with a 
mental health practitioner within the specified time period. For each 
denominator event (discharges), the follow-up visit must occur after the 
applicable discharge. An outpatient visit on the date of discharge should be 
included in the measure. 


30-day follow-up An outpatient follow-up encounter with a mental health practitioner up to 30 
days after hospital discharge. To identify outpatient follow-up encounters, 
use the CPT codes or the UB-92 revenue codes in Table FUH-B. 


7-day follow-up An outpatient follow-up encounter with a mental health practitioner up to 7 
days after hospital discharge. To identify outpatient follow-up encounters, 
use the CPT codes or the UB-92 revenue codes in Table FUH-B. 


III. Hybrid Specification 


None. 


Table FUH-B: Codes to Identify Outpatient Mental Health Encounters or Intermediate Treatment 


Description UB-92 Revenue * 


Outpatient or 90801, 90802, 90804-90819, 90821-90824, G0155, G0176, G0177, 0513, 0900, 0901, 
intermediate care 90826-90829, 90845, 90847, 90849, 90853, H0002, H0004, H0031, 0905-0907, 0909-0916, 


90857, 90862, 90870, 90875-90876, 99201- | H0034-H0037, H0039, 0961 
99205, 99211-99215, 99241-99245, 99341- H0040, H2000, H2001, 
99345, 99347-99350, 99383-99387, 99393- H2010-H2020, M0064, 
99397, 99401-99404, 99510 $9480, $9484, $9485 


*The MCO does not need to determine practitioner type for follow-up visits identified through UB-92 Revenue codes. 
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An MCHP that submits HEDIS data to NCQA must provide the following data elements: 


Table 3 - Data Elements for Follow-Up After Hospitalization for Mental Illness (FUH) 
Administrative 


Measurement year 
Data collection methodology (administrative) 
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HEDIS 2013 ANNUAL DENTAL VISIT (ADV) 


The following is the definition of the Annual Dental Visit measure, an Effectiveness of Care measure, 


and the specific parameters as defined by the NCQA. 


The percentage of enrolled members 2—21 years of age who had at least one dental visit 
during the measurement year. This measure applies only if dental care is a covered benefit 


in the MCO’s Medicaid contract. 


Table 4 - HEDIS 2013 Technical Specifications for Annual Dental Visit (ADV) 
I. Eligible Population 


Product line Medicaid. 
Ages 2-21 years as of December 31 of the measurement year. The measure is 
reported for each of the following age stratifications and as a combined rate. 
e 2-3-years e 11-14-years e 19-21-years 
e 4-6-years e 15—18-years e Total 
e 7-10-years 
Continuous The measurement year. 
enrollment 
Allowable gap No more than 1 gap in enrollment of up to 45 days during the measurement year. 
To determine continuous enrollment for a Medicaid beneficiary for whom 
enrollment is verified monthly, the member may not have more than a 1-month 
gap in coverage (i.e., a member whose coverage lapses for 2 months [60 days] is 
not considered continuously enrolled). 
Anchor date December 31 of the measurement year. 
Benefit Dental. 


Event/diagnosis None. 


II. Administrative Specification 





Denominator The eligible population for each age group and the combined total. 


Numerator One or more dental visits with a dental practitioner during the measurement year. 
A member had a dental visit if a submitted claim/encounter contains any of the 
codes in Table ADV-A. 


III. Hybrid Specification 


None. 
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Table ADV-A: Codes to Identify Annual Dental Visits 
CPT | HCPCSICDT-3 ICD-9-CM Procedure 


D8010-D8999, D9110-D9999 
Note: Current Dental Terminology (CDT) is the equivalent dental version of the CPT physician procedural coding system. 





70300, 70310, 70320, D0120-D0999, D1110-D2999, D3110-D3999, D4210- | 23, 24, 87.11, 87.12, 89.31, 93.55, 96.54, 97.22, 
70350, 70355 D4999, D5110-D5899, D6010-D6205, D7111-D7999, | 97.33-97.35, 99.97 


An MCHP that submits HEDIS data to NCQA must provide the following data elements: 


Table 5 - Data Elements for Annual Dental Visits 
Administrative 


Data collection methodology (administrative) 
Eligible population For each age stratification and total 


Numerator events by administrative data For each age stratification and total 
Reported rate For each age stratification and total 
Lower 95% confidence interval For each age stratification and total 


Upper 95% confidence interval For each age stratification and total 
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3.2 Methods of Calculating Performance Measures 


The HEDIS technical specifications allow for two methods of calculating performance measures: |) 
the Administrative Method and 2) the Hybrid Method. Each year one of the measures selected for 
this review, allows for Administrative or Hybrid methods of review. The two remaining measures 


are each calculated using the Administrative Method only. 


The Administrative Method involves examining claims and other databases (administrative data) to 
calculate the number of members in the entire eligible population who received a particular service 
(e.g., well-child visits). The eligible population is defined by the HEDIS technical specifications. 
Those cases in which administrative data show that the member received the service(s) examined 
are considered “hits” or “administrative hits.” The HEDIS technical specifications provide 


acceptable administrative codes for identifying an administrative hit. 


For the Hybrid Method, administrative data are examined to select members eligible for the 
measure. From these eligible members, a random sample is taken from the appropriate 
measurement year. Members in the sample are identified who received the service(s) as evidenced 
by a claim submission or through external sources of administrative data (e.g., State Public Health 
Agency Vital Statistics or Immunization Registry databases). Those cases in which an administrative 
hit cannot be determined are identified for further medical record review. Documentation of all or 
some of the services in the medical record alone or in combination with administrative data is 


considered a “hybrid hit.” 


Administrative hits and hybrid hits are then summed to form the numerator of the rate of members 
receiving the service of interest (e.g., appropriate doctor’s visit). The denominator of the rate is 
represented by the eligible population (administrative method) or those sampled from the eligible 
population (hybrid method). A simple formula of dividing the numerator by the denominator 
produces the percentage (also called a “rate”) reported to the SMA and the SPHA. 

Additional guidance is provided in the HEDIS Technical Specifications: Volume 23 for appropriate 
handling of situations involving oversampling, replacement, and treatment of contraindications for 


services. 





3 National Committee for Quality Assurance. HEDIS 2013, Volume 2: Technical Specifications. Washington, 
D.C.: NCQA. 
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TIME FRAME 

The proper time frame for selection of the eligible population for each measure is provided in the 
HEDIS technical specifications. For the measures selected, the “measurement year” referred to 
calendar year prior to the review year. All events of interest (e.g. follow-up visits) must also have 


occurred during the calendar year prior to the review year. 


PROCEDURES FOR DATA COLLECTION 

The HEDIS technical specifications for each measure validated are reviewed by the EQRO Project 
Director and the EQRO Research Analyst. Extensive training in data management and programming 
for Healthcare quality indices, clinical training, research methods, and statistical analysis expertise 
were well represented among the personnel involved in adapting and implementing the Validating of 
Performance Measures Protocol to conform to the HEDIS, SMA, and SPHA requirements while 
maintaining consistency with the Validating Performance Measures Protocol. The following sections 
describe the procedures for each activity in the Validating Performance Measures Protocol as they 


were implemented for the HEDIS measures validated. 


Pre-On-Site Activity One: Reviewer Worksheets 
Reviewer Worksheets are developed for the purpose of conducting activities and recording 
observations and comments for follow-up at the site visits. These worksheets are reviewed and 
revised to update each specific item with the current year’s HEDIS technical specifications. Project 
personnel meet regularly to review available source documents and develop the Reviewer 
Worksheets for conducting pre-on-site, on-site, and post-on-site activities as described below. 
These reviews formed the basis for completing the CMS Protocol Attachments (V, VII, X, XII, XIll, 
and XV) of the Validating Performance Measures Protocol for each measure and MCHP. Source 
documents used to develop the methods for review and complete the Attachments included the 
following pertinent to the current review year: 

e HEDIS Data Submission Tool (DST) 

e HEDIS Road Map 

e HEDIS Audit Report 

e HEDIS SPHA Reports 
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Pre-On-Site Activity Two: Preparation of MO HealthNet MCOs 

Orientation teleconferences with each MO HealthNet MCHP are conducted annually by the EQRO. 
The purpose of this orientation conference is to provide education about the Validating 
Performance Measures protocol and the EQRO’s submission requirements. All written materials, 
letters and instructions used in the orientation are reviewed and approved by the SMA in advance. 
Prior to the teleconference calls, the MCHPs are provided information on the technical objectives, 
methods, procedures, data sources, and contact information for EQRO personnel. The MCHPs 
were requested to have the person(s) responsible for the calculation of that year’s HEDIS 
performance measures to be validated in attendance. Teleconference meetings were led by the 
EQRO Project Director, with key project personnel and a representative from the SMA in 
attendance. Provided via the teleconferences is technical assistance focused on describing the 
Validating Performance Measures Protocol; identification of the three measures selected for 
validation each year; the purpose, activities and objectives of the EQRO; and definitions of the 
information and data needed for the EQRO to validate the performance measures. All MCHP 
questions about the process are answered at this time and identified for further follow-up by the 
EQRO if necessary. In addition to these teleconference calls, presentations and individual 
communications with personnel at MCHPs responsible for performance measure calculation are 


conducted. 


Formal written requests for data and information for the validation of performance measures are 
submitted to the MCHPs by the EQRO recognizing the need to provide adequate time for data and 
medical record collection by each MCHP. This information is returned to the EQRO within a 
specific time frame (see Appendix 3). A separate written request is sent to the MCHPs requesting 
medical records be submitted to the EQRO for a sample of cases. These record requests are then 
submitted by the providers to the EQRO. Detailed letters and instructions are mailed to QI/UM 
Coordinators and MCHP Administrators explaining the type of information, purpose, and format of 
submissions. EQRO personnel are available and respond to electronic mail and telephone inquiries 


and any requested clarifications throughout the evaluation process. 
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The following are the data and documents requested from MCHPs for the Validating Performance 


Measures Protocol: 





HEDIS Data Submission Tool for all three measures for the MO HealthNet Managed Care 
Population only. 

Prior year’s HEDIS Audit Report. 

HEDIS RoadMap for the previous HEDIS year. 

List of cases for denominator with all appropriate year’s HEDIS data elements specified in 
the measures. 

List of cases for numerators with all appropriate year’s HEDIS data elements specified in the 
measures, including fields for claims data and all other administrative data used. 

All worksheets, memos, minutes, documentation, policies and communications within the 
MCHP and with HEDIS auditors regarding the calculation of the selected measures. 

List of cases for which medical records are reviewed, with all required HEDIS data elements 
specified in the measures. 

Sample medical record tools used for hybrid methods for the three HEDIS measures for the 
MO HealthNet Managed Care population; and instructions for reviewers. 

Policies, procedures, data and information used to produce numerators and denominators. 
Policies, procedures, and data used to implement sampling (if sampling was used). Ata 
minimum, this should include documentation to facilitate evaluation of: 

© Statistical testing of results and any corrections or adjustments made after 
processing. 

o Description of sampling techniques and documentation that assures the reviewer 
that samples used for baseline and repeat measurements of the performance 
measures are chosen using the same sampling frame and methodology. 

o Documentation of calculation for changes in performance from previous periods (if 
comparisons were made), including tests of statistical significance. 

Policies and procedures for mapping non-standard codes, where applicable. 

Record and file formats and descriptions for entry, intermediate, and repository files. 
Electronic transmission procedures documentation. (This will apply if the MCHP sends or 
receives data electronically from vendors performing the HEDIS abstractions, calculations or 
data entry) 

Descriptive documentation for data entry, transfer, and manipulation programs and 


processes. 
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e Samples of data from repository and transaction files to assess accuracy and completeness 
of the transfer process. 

e Documentation of proper run controls and of staff review of report runs. 

e Documentation of results of statistical tests and any corrections or adjustments to data 
along with justification for such changes. 

e Documentation of sources of any supporting external data or prior years’ data used in 
reporting. 

e Procedures to identify, track, and link member enrollment by product line, product, 
geographic area, age, sex, member months, and member years. 

e Procedures to track individual members through enrollment, disenrollment, and possible re- 
enrollment. 

e Procedures used to link member months to member age. 

e Documentation of “frozen” or archived files from which the samples were drawn, and if 
applicable, documentation of the MCHP’s process to re-draw a sample or obtain necessary 
replacements. 

e Procedures to capture data that may reside outside the MCHP’s data sets (e.g. MOHSAIC). 

e Policies, procedures, and materials that evidence proper training, supervision, and adequate 
tools for medical record abstraction tasks. (May include training material, checks of inter- 
rater reliability, etc.) 

e Appendix V — Information Systems Capabilities Assessment for Managed Care Organizations 


and Prepaid Health Plans 


Pre-On-Site Activity Three: Assess the Integrity of the MCHP's Information System 
The objective of this activity is to assess the integrity of the MCHPs’ ability to link data from 
multiple sources. All relevant documentation submitted by the MCHPs is reviewed by EQRO 
personnel. The review protocols require that an Information Systems Capability Assessment (ISCA) 
be administered every other year. The EQRO follows this process and the MCHPs are informed if 
a full ISCA review will occur when the Orientation Conference Calls occur. The results of this 
review are reflected in the final EQRO. EQRO personnel also review HEDIS RoadMap submitted 


by each MCHP. Detailed notes and follow-up questions are formulated for the site visit reviews. 
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On-Site Activity One: Assess Data Integration and Control 
The objective of this activity is to assess the MCHPs’ ability to link data from multiple sources and 
determine whether these processes ensure the accurate calculation of the measures. A series of 
interviews and in-depth reviews are conducted by the EQRO with MCHP personnel (including both 
management and technical staff and 3'4 party vendors when applicable). These site visit activities 
examine the development and production procedures of the HEDIS performance measures and the 
reporting processes, databases, software, and vendors used to generate these rates. This includes 
reviewing data processing issues for generating the rates and determining the numerator and 
denominator counts. Other activities involve reviewing database processing systems, software, 
organizational reporting structures, and sampling methods. The following are the activities 
conducted at each MCHP: 

e Review results of run queries (on-site observation, screen-shots, test output) 

e Examination of data fields for numerator & denominator calculation (examine field 

definitions and file content) 

e Review of applications, data formats, flowcharts, edit checks and file layouts 

e Review of source code, software certification reports 

e Review HEDIS repository procedures, software manuals 

e Test for code capture within system for measures (confirm principal & secondary codes, 

presence/absence of non-standard codes) 
e Review of operating reports 
e Review information system policies (data control, disaster recovery) 


e Review vendor associations & contracts 


The following are the type of interview questions developed for the site visits: 
e What are the processes of data integration and control within information systems? 
e What documentation processes are present for collection of data, steps taken and 
procedures to calculate the HEDIS measures? 
e What processes are used to produce denominators? 
e What processes are used to produce numerators? 
e How is sampling done for calculation of rates produced by the hybrid method? 


e How does the MCHP submit the requirement performance reports to the State? 
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From the site visit activities, interviews, and document reviews, Attachment V (Data Integration and 
Control Findings) of the CMS Protocol is completed for each MCHP and performance measure 


validated. 


On-Site Activity Two: Assess Documentation of Data and Processes Used to Calculate 
and Report Performance Measures 


The objectives of this activity are to assess the documentation of data collection, assess the process 
of integrating data into a performance measure set, and examine procedures used to query the data 


set to identify numerators, denominators, generate a sample, and apply proper algorithms. 


From the site visit activities, interviews, review of numerator and denominator files and document 
reviews, Attachment VII (Data and Processes Used to Calculate and Report Performance) of the 
CMS Protocol is completed for each MCHP and measure validated. One limitation of this step is 
the inability of the MCHPs to provide documentation of processes used to calculate and report the 
performance measures due to the use of proprietary software or off-site vendor software and 
claims systems. However, all MCHPs are historically able to provide documentation and flow- 
charts of these systems to illustrate the general methods employed by the software packages to 


calculate these measures. 


On-Site Activity Three: Assess Processes Used to Produce the Denominators 
The objectives of this activity are to: |) determine the extent to which all eligible members are 
included; 2) evaluate programming logic and source codes relevant to each measure; and 3) evaluate 


eligibility, enrollment, age, codes, and specifications related to each performance measure. 


The content and quality of the data files submitted are reviewed to facilitate the evaluation of 
compliance with the HEDIS 2013 technical specifications. The MCHPs consistently submit the 
requested level of data (e.g., all elements required by the measures or information on hybrid or 
administrative data). In order to produce meaningful results, the EQRO requires that all the 


MCHPs submit data in the format requested 


From the site visit activities, interviews, review of numerator and denominator files and document 
reviews, Attachment X (Denominator Validation Findings) of the CMS Protocol is completed for 


each MCHP and the performance measures being validated. 


Performance Management Solutions Group 39 
A division of Behavioral Health Concepts, Inc. 





MO HealthNet Managed Care External Quality Review Section 3 
Supplemental Report — 2013 Performance Measures 


On-Site Activity Four: Assess Processes Used to Produce the Numerators 

The objectives of this activity are to: |) evaluate the MCHPs’ ability to accurately identify medical 
events (e.g., appropriate doctor’s visits); 2) evaluate the MCHPs’ ability to identify events from other 
sources (e.g., medical records, State Public Immunization Registry); 3) assess the use of codes for 
medical events; 4) evaluate procedures for non-duplication of event counting; 5) examine time 
parameters; 6) review the use of non-standard codes and maps; 7) identify medical record review 
procedures (Hybrid Method); and 8) review the process of integrating administrative and medical 


record data. 


Validation of the numerator data for all three measures is conducted using the parameters specified 
in the HEDIS Technical Specifications; these parameters applied to dates of service(s), diagnosis 
codes, and procedure codes appropriate to the measure in question. For example, the Annual 
Dental Visit measure requires that all dates of service occurred between January | and December 
3 lof the review year. Visits outside this valid date range were not considered. Similar validation is 
conducted for all three measures reviewed. This numerator validation is conducted on either all 
numerator cases (Administrative Method) or on a sample of cases (Hybrid Method). 

Additional validation for measures being calculated using the Hybrid Method is conducted. The 
Protocol requires the EQRO to sample up to 30 records from the medical records reported by the 
MCHP as meeting the numerator criteria (hybrid hits). In the event that the MCHP reports fewer 
than 30 numerator events from medical records, the EQRO requests all medical records that are 


reported by the MCHP as meeting the numerator criteria. 


Initial requests for documents and data are made on early in the calendar year with submissions due 
approximately six weeks later. The EQRO requires the MCHPs to request medical records from 
the providers. The MCHPs are given a list of medical records to request, a letter from the State 
explaining the purpose of the request, and the information necessary for the providers to send the 
medical records directly to the EQRO. The submission deadline is determined based on the 
original request date, and the date of the final receipt based on that date. The record receipt rate is 


historically excellent. In recent years the EQRO has received 100% of records requested. 
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The review of medical records is conducted by experienced RNs currently licensed and practicing in 
the State of Missouri. These RNs participate in the training and medical record review process. 
They are required to have substantive experience conducting medical record reviews for HEDIS 


measures. 


A medical record abstraction tool for the HEDIS measures to be reviewed is developed by the 
EQRO Project Director and revised in consultation with a nurse consultant, the EQRO Research 
Analyst, and with the input from the nurse reviewers. The HEDIS technical specifications and the 
Validating Performance Measures Protocol criteria are used to develop the medical record review 
tools and data analysis plan. A medical record review manual and documentation of ongoing 
reviewer questions and resolutions were developed for the review. A half day of training is 
conducted annually by the EQRO Project Director and staff, using sample medical record tools and 
reviewing all responses with feedback and discussion. The reviewer training and training manual 
covered content areas such as Health Insurance Portability and Accountability Act (HIPAA), 
confidentiality, conflict of interest, review tools, and project background. Teleconference meetings 
between the nurses, coders, and EQRO Project Director are conducted as needed to resolve 


questions and coding discrepancies throughout the duration of the medical record review process. 


A data entry format with validation parameters was developed for accurate medical record review 
data entry. The final databases are reviewed for validity, verified, and corrected prior to performing 
analyses. All data analyses are reviewed and analyzed by the EQRO Project Director. CMS 
Protocol Attachments XII (Impact of Medical Record Findings) and XIll (Numerator Validation 


Findings) are completed based on the medical record review of documents and site visit interviews. 
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On-Site Activity Five: Assess Sampling Process (Hybrid Method) 
The objective of this activity is to assess the representativeness of the sample of care provided. 
e Review HEDIS RoadMap 
e Review Data Submission Tool (DST) 
e Review numerator and denominator files 
e Conduct medical record review for measures calculated using hybrid methodology 
e Determine the extent to which the record extract files are consistent with the data found in 
the medical records 
e Review of medical record abstraction tools and instructions 


e Conduct on-site interviews, activities, and review of additional documentation 


For those MCHPs that calculating one of the identified HEDIS measures via the hybrid methodology, 
a sample of medical records (up to 30) is conducted to validate the presence of an appropriate well- 


child visit that contributed to the numerator. 


On-Site Activity Six: Assess Submission of Required Performance Measures to State 
The objective of this activity is to assure proper submission of findings to the SMA and SPHA. The 
DST is obtained from the SPHA to determine the submission of the performance measures 
validated. Conversations with the SPHA representative responsible for compiling the measures for 
all MCHPs in the State occurred with the EQRO Project Director to clarify questions, obtain data, 


and follow-up on MCHP submission status. 


Post- On-Site Activity One: Determine Preliminary Validation Findings for each 
Measure 


Calculation of Bias 

The CMS Validating Performance Measures Protocol specifies the method for calculating bias based 
on medical record review for the Hybrid Method. In addition to examining bias based on the 
medical record review and the Hybrid Method, the EQRO calculates bias related to the 
inappropriate inclusion of cases with administrative data that fall outside the parameters described in 
the HEDIS Technical Specifications. For measures calculated using the Administrative Method, the 
EQRO examines the numerators and denominators for correct date ranges for dates of birth and 


dates of service as well as correct enrollment periods and codes used to identify the medical events. 
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This is conducted as described above under on-site activities three and four. The estimated bias in 
the calculation of the HEDIS measures for the Hybrid Method is calculated using the following 
procedures, methods and formulas, consistent with the Validating Performance Measures Protocol. 


Specific analytic procedures are described in the following section. 


Analysis 

Once the medical record review is complete, all administrative data provided by the MCHPs in their 
data file submissions for the HEDIS hybrid measure are combined with the medical record review 
data collected by the EQRO. This allows for calculation of the final rate. In order for each event to 
be met, there must be documented evidence of an appropriate event code as defined in the HEDIS 


Technical Specifications. 


For the calculation of bias based on medical record review for the MCHPs using the Hybrid Method 
for the HEDIS measure selected, several steps are taken. First, the number of hits based on the 
medical record review is reported (Medical Records Validated by EQRO). Second, the Accuracy 
(number of Medical Records able to be validated by EQRO/total number of Medical Records 
requested by the EQRO for audit) and Error Rates (100% - Accuracy Rate) are determined. Third, 
a weight for each Medical Record is calculated (100%/denominator reported by the MCHP) as 
specified by the Protocol. The number of False Positive Records is calculated (Error Rate * 
numerator hits from Medical Records reported by the MCHP). This represents the number of 
records that are not able to be validated by the EQRO. The Estimated Bias from Medical Records is 


calculated (False Positive Rate * Weight of Each Medical Record). 


To calculate the Total Estimated Bias in the calculation of the performance measures, the 
Administrative Hits Validated by the EQRO (through the previously described file validation 
process) and the Medical Record Hits Validated by the EQRO (as described above) are summed and 
divided by the total Denominator reported by the MCHP on the DST to determine the Rate 
Validated by the EQRO. The difference between the Rate Validated by the EQRO and the Rate 
Reported by the MCHP to the SMA and SPHA is the Total Estimated Bias. A positive number 
reflects an overestimation of the rate by the MCHP, while a negative number reflects an 


underestimation. 


Once the EQRO concludes its on-site activities, the validation activity findings for each performance 


measure are aggregated. This involves the review and analysis of findings and Attachments produced 
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for each performance measure selected for validation and for the MCHP’s Information System as a 
result of pre-on-site and on-site activities. The EQRO Project Director reviews and finalizes all 
ratings and completed the Final Performance Measure Validation Worksheets for all measures 
validated for each of the MCHPs. Ratings for each of the Worksheet items (0 = Not Met; | = 
Partially Met; 2 = Met) are summed for each worksheet and divided by the number of applicable 
items to form a rate for comparison to other MCHPs. The worksheets for each measure are 


examined by the EQRO Project Director to complete the Final Audit Rating. 


Below is a summary of the final audit rating definitions specified in the Protocol. Any measures not 
reported are considered “Not Valid.” A Total Estimated Bias outside the 95% upper or lower 


confidence limits of the measures as reported by the MCHP on the DST is considered “Not Valid”. 


Fully Compliant: Measure was fully compliant with State (SMA and SPHA) 
specifications. 


Substantially Measure was substantially compliant with State (SMA and SPHA) 
Compliant: specifications and had only minor deviations that did not 
significantly bias the reported rate. 


Not Valid: Measure deviated from State specifications such that the reported 
rate was significantly biased. This designation is also assigned to 
measures for which the data provided to the EQRO could not be 
independently validated. 


‘Significantly Biased’ was defined by the EQRO as being outside 
the 95% confidence interval of the rate reported by the MCHP on 
the HEDIS 2007 Data Submission Tool. 
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PLANNING COMPLIANCE MONITORING ACTIVITIES 

Gathering Information on the MO HealthNet MCHP Characteristics 

Currently there are three MCHPs contracted with the State Medicaid Agency (SMA) to provide 
MO HealthNet Managed Care in three Regions of Missouri. The Eastern Region includes St. 
Louis City, St. Louis County, and twelve surrounding counties. The Western Region includes 
Kansas City/Jackson County and twelve surrounding counties. The Central Region includes 
twenty-eight counties in the center of the state. All three MCHPs serve MO HealthNet 
members in all three regions, these three MCHPs are: Missouri Care (MOCare), Home State 


Health Plan (Home State), and Healthcare USA (HCUSA). 


Determining the Length of Visit and Dates 

On-site compliance reviews are conducted in two days at each MCHP, with several reviewers 
conducting interviews and activities concurrently. Document reviews occur prior to the 
complete on-site review at all MCHPs. Document reviews and the Validation of Performance 
Measures interviews are conducted on the first day of the on-site review. Interviews, 
presentations, and additional document reviews are scheduled throughout the second day, 
utilizing all team members for Validating Performance Improvement Projects, and Monitoring 
Medicaid Managed Care Organizations (MCHPs) and Prepaid Inpatient Health Plans (PIHPs). 
The time frames for on-site reviews are determined by the EQRO and approved by the SMA 


before scheduling each MCHP. 


Establishing an Agenda for the Visit 

An agenda is developed to maximize the use of available time, while ensuring that all relevant 
follow-up issues are addressed. A sample schedule is developed that specifies times for all 
review activities including the entrance conference, document review, Validating Performance 
Improvement Project evaluation, Validating Performance Measures review, conducting the 
interviews for the Compliance Protocol, and the exit conference. A coordinated effort with 
each MCHP occurs to allow for the most effective use of time for the EQRO team and MCHP 
staff. The schedule for the on-site reviews is approved by the SMA in advance and forwarded to 


each MCHP to allow them the opportunity to prepare for the review. 
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Providing Preparation Instructions and Guidance to the MO HealthNet MCOs 

A letter (see Appendix 12) is sent to each MCHP indicating the specific information and 
documents required on-site, and the individuals requested to attend the interview sessions. The 
MCHPs schedule their own staff to ensure that appropriate individuals are available and that all 


requested documentation is present during the on-site review day. 


OBTAINING BACKGROUND INFORMATION FROM THE STATE MEDICAID 
AGENCY 


Interviews and meetings occur with individuals from the SMA to prepare for the on-site review, 
and obtain information relevant to the review prior to the on-site visits. The Compliance 
Review team members request the contract compliance documents prepared annually by the 
SMA. The information on MCHP compliance with the current MO HealthNet Managed Care 
contract is reviewed, along with required annual submission and approval information. This 
documentation is used as a guide for the annual review although final compliance with state 
contract requirements is determined by the SMA. These determinations are utilized in assessing 
compliance with the Federal Regulations. All documentation gathered by the SMA is clarified and 
discussed to ensure that accurate interpretation of the SMA findings is reflected in the review 
comments and findings. SMA expectations, requirements, and decisions specific to the MO 


HealthNet Managed Care Program are identified during these discussions. 


DOCUMENT REVIEW 

Documents chosen for review are those that best demonstrate each MCHP’s ability to meet 
federal regulations. Certain documents, such as the Member Handbook, provide evidence of 
communication to members about a broad spectrum of information including enrollee rights and 
the grievance and appeal process. Provider handbooks are reviewed to ensure that consistent 
information is shared regarding enrollee rights and responsibilities. SMA MO HealthNet 
Managed Care contract compliance worksheets, and specific policies that are reviewed annually 
or that are yet to be approved by the SMA, are reviewed to verify the presence or absence of 
evidence that required written policies and procedures exist meeting federal regulations. Other 
information, such as the Annual Quality Improvement Program Evaluation is requested and 
reviewed to provide insight into the MCHP’s compliance with the requirements of the SMA 
Quality Improvement Strategy, which is an essential component of the MO HealthNet Managed 


Care contract, and is required by the federal regulations. MCHP Quality Improvement 
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Committee meeting minutes are reviewed. 


Case Management and Member Services policies and instructions, as well as training curriculum 
are often reviewed to provide insight into the MCHP’s philosophy regarding case management 
activities. In addition interviews, based on questions from the SMA and specific to each MCHP’s 
Quality Improvement Evaluation, are conducted with direct services staff and administrative staff 
to ensure that local procedures and practices corresponded to the written policies submitted 
for approval. When it is found that specific regulations are “Partially Met,” additional documents 
are requested of each MCHP. In addition, interview questions are developed for identified staff 
to establish that practice directly with members reflects the MCHPs’ written policies and 
procedures. Interviews with Administrative staff occur to address the areas for which 
compliance is not fully established through the pre-site document review process, and to clarify 


responses received from the staff interviews. 


The following documents were reviewed for all MO HealthNet MCHPs: 
e Annual State contract compliance ratings; 
e Results, findings, and follow-up information from the previous External Quality Review; 
and 


e Annual MO HealthNet MCHP Evaluation, submitted each spring. 


CONDUCTING INTERVIEWS 

After discussions with the SMA, the focus of that year’s Compliance Review is determined. This 
often results in in-depth interviews with Member Services and Case Management Staff. The goal 
of these interviews is to validate that practices at the MCHPs, particularly those directly 
affecting members’ access to quality and timely health care, are in compliance with approved 
policies and procedures. The interview questions are developed using the guidelines available in 
the Compliance Protocol, are focused on areas of concern based on each MCHP’s Annual 
Evaluation, or address issues of concern expressed by the SMA. Interviews conducted with 
administrative and management level MCHP staff, enable reviewers to obtain a clearer picture of 
the degree of compliance achieved through policy implementation. Corrective action taken by 
each MCHP is determined from previous years’ reviews. This process reveals a wealth of 
information about the approach each MCHP is using to become compliant with federal 


regulations. The current process of a document review, supported by interviews with front line 


Performance Management Solutions Group 49 
A division of Behavioral Health Concepts, Inc. 





MO HealthNet Managed Care External Quality Review Compliance with Regulations 
Supplemental Report — 2013 


and administrative staff, is developed to provide evidence of a system that delivers quality and 
timely services to members, and the degree to which appropriate access was available. The 
interviews provide reviewers with the opportunity to explore issues not addressed in the 
documentation. Additionally, this approach continues to provide follow-up from previous 
EQRO evaluations. A site visit questionnaire for direct services staff, and a separate interview 
tool for Administrators is developed for each MCHP annually. The questions seek concrete 
examples of activities and responses that validate that these activities are compliant with 


contractual requirements and federal regulations. 


COLLECTING ACCESSORY INFORMATION 

Additional information used in completing the compliance determination included: discussions 
with the EQR reviewers and MO HealthNet MCHP QI/UM staff regarding management 
information systems; Validating Performance Measures; and Validating Performance 
Improvement Projects. The review evaluates information from these sources to validate MCHP 
compliance with the pertinent regulatory provisions within the Compliance Protocol. These 
findings are documented in the EQR final report and are also reflected in rating 


recommendations. 


ANALYZING AND COMPILING FINDINGS 

The review process includes gathering information and documentation from the SMA about 
policy submission and approval, which directly affects each MCHP’s contract compliance. This 
information is analyzed to determine how it relates to compliance with the federal regulations. 
Next, interview questions are prepared, based on the need to investigate if practice exists in 
areas where approved policy is not available, and if local policy and procedures are in use when 
approved policy is not complete. The interview responses and additional documentation 
obtained on-site are then analyzed to evaluate how they contributed to each MCHP’s 
compliance. All information gathered is assessed, re-reviewed and translated into 
recommended compliance ratings for each regulatory provision. This information is recorded 
on the MO HealthNet Managed Care scoring form and can be found in the protocol specific 


sections of this section of the report. 


REPORTING TO THE STATE MEDICAID AGENCY 
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During the meetings with the SMA following the on-site review, preliminary findings and 
comparisons to the previous ratings are presented. Discussion occurs with the SMA staff to 
ensure that the most accurate information is available and to confirm that a sound rationale is 
used in rating determinations. The SMA approves the process and allows the EQRO to finalize 
the ratings for each regulation. Sufficient detail is included in all worksheets to substantiate any 


rating lower than “Met.” The actual ratings are included in the final report. 


COMPLIANCE RATINGS 

All information gathered prior to the compilation of the final report is utilized is compiling the 
final ratings. This includes the most up-to-date results of MCHP submissions to the SMA of 
policy and procedures that meet or exceed contract compliance. This information is then 
compared to the requirements of the each federal regulation to ensure that policy and practice 
are in compliance. The SMA has provided ongoing approval to the EQRO to utilize the 
Compliance Rating System developed during the previous reviews. This system is based ona 
three-point scale (“Met,” Partially Met,” “Not Met”) for measuring compliance, as determined by 
the EQR analytic process. The determinations found in the Compliance Ratings considered 
SMA contract compliance, review findings, MCHP policy, ancillary documentation, and staff 
interview summary responses that validate MCHP practices observed on-site. In some instances 
the SMA MO HealthNet Managed Care contract compliance tool rates a contract section as 
“Met” when policies are submitted, even if the policy has not been reviewed and “finally 
approved.” If the SMA considers the policy submission valid and ratesit as “Met,” this rating is 
used unless practice or other information calls this into question. If this conflict occurs, it is 
explained in the final report documentation. The scale allows for credit when a requirement is 


Partially Met. Ratings were defined as follows: 


All documentation listed under a regulatory provision, or one of its 
components was present. MCHP staff was able to provide responses 
to reviewers that were consistent with one another and the available 
documentation. Evidence was found and could be established that 
the MCHP was in full compliance with regulatory provisions. 


Partially Met: There was evidence of compliance with all documentation 
requirements, but staff was unable to consistently articulate 
processes during interviews; or documentation was incomplete or 
inconsistent with practice. 


Not Met: Incomplete documentation was present and staff had little to no 
knowledge of processes or issues addressed by the regulatory 
provision. 
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Solutions Group 

Division of Behavioral Health Concepts, inc. BHC. 





Orientation Agenda 












Introductions 
Orientation to Technical Methods and 
Objectives of Protocols 
Review of Information, Data Requests, and 
Timeframes 
Performance Measures 
Performance Improvement Projects 
Case Management Special Project 
Compliance and Site Visits 
Closing Comments, Questions 
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Materials Provided 







Objectives and Technical Methods 
Validation of Performance Measures 
Validation of Encounter Data 
Validation of Performance Improvement Projects 
Health Plan Compliance 

Requests for information and data 

List of BHC contacts for each protocol 


Presentation 
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Overview 






Protocol Activities 
Information and Data Requests 


Contact Persons 
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“ — Validation of 
LESS —-—- _/ Performance Measures 


HEDIS 2013 Measure Validation 

» Annual Dental Visit 

» Childhood Immunization Status (Combo 3) 
Follow-Up After Hospitalization for Mental 
Illness 

» Administrative 

Hybrid method 
Review up to 30 medical records per measure 
sampled randomly 
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Case Management 
Special Project 


Cases will be reviewed in regards to current 
MHD contract requirements 


| A, Performance Management 
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Assessment 

Care Plan 

Discharge 

Transition of Care (when applicable) 
Case Review Tool 


Specific by case type: i.e. Lead, Prenatal, 
Disease Management... 


Performance Management Solutions Group 
A division of Behavioral Health Concepts, Inc. 





6l 


MO HealthNet Managed Care External Quality Review Appendix | 
Supplemental Report — 2013 MCHP Orientation PowerPoint Slides 







Purpose and 
Objectives 


/\| Performance Management 
Solutions Group 
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To assess the completeness of Case 
Management Records. 


To validate the health plans’ compliance 
with MHD contract requirements for Case 
Management. 


To examine the match between Health Plan 
enrollees in Case Management and those 
enrollees known to MHD that meet Case 
Management criteria. 
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Medical Record 
Reviews 
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HEDIS 
Medical record samples requested from 
Health Plans for 1 possible hybrid measure (N 
< 30 per measure; 4 weeks) 


Case Management Special Project 


Medical records samples requested from Health Plans 
(N > 30; 4 weeks and onsite) 
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Medical Record 
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Solutions Group 
a Division of Behavioral Health Concepts, Inc. BHC. 







Reviewed and abstracted by experienced and 
RNs and Social Workers 


Standard abstraction tools 
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= tiga. alidation of Performance 
“a nieces -w_/ Improvement Projects 






Two Performance Improvement 
Projects underway in 2013 


“One clinical 


One non-clinical 
(Statewide PIP -- ADV) 
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Validation of Performance 
Improvement Projects and 
Submission Requi 










f Performance Management 
Solutions Group 
a Division of Behavioral Health Concepts, Inc. BHC. 






PIP Checklist Elements 
Project narratives, baseline measures, methods, interventions, and planned analyses. 
Examples of information are contained in the CMS protocol, Validation of Performance 
Measures/[1] 
Phase-in/timeframe for each phase of each PIP[1] 
Problem identification 
Hypotheses 
Evaluation Questions 
Description of intervention(s) 
Methods of sampling, measurement 
Planned analyses 
Sample tools, measures, surveys, etc. 
Baseline data source and data 
Cover letter with clarifying information 
ye Raw data files (if applicable, on-site) 
Medical records or other original data sources (if applicable, on-site) 
Additional data as needed 


[1] U.S. Department of Health and Human Services, Centers for Medicare & Medicaid 

Services (2002) VALIDATING PERFORMANCE IMPROVEMENT PROJECTS A protocol 

ie use ut een Medicaid External Quality Review Activities: Final Protocol Version 
.O May 1, 
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— Health Plan 


Sehitions Group 
‘ 


“=——-s« _/ Compliance 
Follow up Compliance review year 
Enrollee Rights 
Provider Networks/Directories 
' Transition of Care 
Health Homes 


Fraud and Abuse 
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Serene _) Site Visits 


an =| 
Target for late June 2014 


» MO Care June 1/7 & 18 

» HCUSA June 24 & 25 
Home State June 24 & 26 

« Health Plan Compliance Reviews 
On-site activities 

» Performance Measure Validation 

» Performance Improvement Project Validation 
Case Management Interviews 
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Final Report 






Health Plan to Health Plan Comparisons: 


Performance Measure audit findings 
and rates 


Performance Improvement Project 
element compliance 


Health Plan Compliance 
Case Management Special Project 
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ete * ~ BHC Team and Coordination 


tat eae mae 
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Appendix 2 — Performance Improvement Project Worksheets 


Performance Improvement Project Validation Worksheet 
\Use this or similar works heet as a guide when validating MCO/PIHP 
Performance Improvement Projects. Answer all questions for each 


‘activity. Refer to protocol for detailed mformation on each area. 





ID ofe valuator Date ofevaluation 
Demographic Information 
MCO/PIHP Name or ID Project Leadey Name Telephone Number 


Name of the Perform ance Improvement Project 





Dates of Study Date Study Initiated 


Type of Delivery System (check all that apply) 


H StaffModel GB Network Director IPA 

HIP A Organization Gimco  PIHP 
Number of Medicaid Enrollees Number Medicare Enro llees in 
inMCO orPIHF* MCO or PIHP 
Number ofMedicaid Enzo llees Total NumberofMCO or PIHP 
in the Study Envyo llees in Study 
Number ofM embers inStudy Fo pulation of Members in 


Sample Frame 


Number ofMCO/PIHP _ Number of MCO/PIHP 
primary care physicians specialty physicians 
Population ofphysicians in Number of physicians in study 


sample frame 


Note: DK = Don'tKnow; NA= Not Applicabk 


* Source: Missouri Medicakl Management Information System COLD Reports, State Sessian MPRI Screen, Revised June 25, 
2004. Enrollment total include enralkes with a future start date; 115, 191.5b, and Title XX1 enrollees as af June 25, 20%04-. 
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Activity 1: ASSESS THE STUDY METHODOLOGY 





Step 1. Review the selected study topics(s) 


ii The wpic was selected through datacollection [Ey Met [Ey Partially met [Ey Not met 
et ge gon eae i eaperts of carole [Ey Not applicahle By Unik to determine 


Topic or problem 
statement 
chnical 
(Prevention of an acute or chronic condition (J High whime services 
(cae foranacute or chionic condition CO High tisk cond itiors 
Nonchnical 
(Process of acoessingor delivering care 
Comments 








2 MCO's/PINP s PIPs, over time, addressed a broad iy Met Gy Partially met fy Not met 


spectrum of key aspects of enrollee cave and services. [i "ke applicadte [iy Unable ta determine 


Project must be clearly focused an identifying and correcting deficiences in care ar 
servicesr ather than on utilization or cost alone. 


Comments 

1.3 MCO's/PINP s PIPs over time, mduded all [my Met [i] Partially met fe] Not met 
enrolled populations: Le., didnot exclude cata 

enrollees such as those with special health care Ey Not applicatie By Unable to determine 
needs. 

Demographic description Age ner 

of MC+ population a Race MC+ 

Comments —____. 
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Step 2: Review the study question(s) 


2.1 Study question(s) stated clearly in writing, my Met [i] Partially met [Hy Not met 
Ol Not applicable ia Unahle to determine 

Stady question(s) as 

stated in narrative: 

Comments 


Step 3. Review selected study indicators(s) 


3-4 The study used objective, clearly defined, Met [i Partially met [Hy Not met 
menrur able Indicators. Not applcahke Unable to determine 
Indicators (let 
Comments 
3-2 Theindicators measured ch amgps im health status, Met [i Partially met [Hy Not met 
function al status or enrollee satisfaction; or process of ; ; . . 
cave with strong association with improved outcomes. [J] *“" #Prlicatile [Ei] ena tn detemmine 
Long term outcomes im plied or stated: Oo ves ONo 
Healhstatu: Satf faction (mem bers): 
Furetional status: Satf faction (prwviders): 


Comments 
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Step 4: Review the identified study population 





4.1 MCO/PINP clearly defined all Medicaid enrollees Met [i] Partially met [i Not met 
to whom the study questions and mdicator's are ; ; ; 
rdevant Oo Not applicable a] Unable to determine 
Demographic description of Age Race — MC+ 
MC+ population sam pled Gender Consnercial 
Didit include: 

1115 pyYe [Na [2] Unable to determine ENA 

1915 Ye [Na [Hy Unable to determine Na 

Chicken in state custody Ye HN habe to determine Na 

Cansert Decee (Western) ve [iN [Hy Lnabee to determine na 

Comments 

4-2 If the MCO /PIH P studied the entire population, did fy Met [Hy Partially met [i Not met 
its data collection approach capture all enrollees w 

whom the study question apphed? [i Not applicahle fi Unable to determine 
Methods of dlentifving § 7 utilizationdata C) wfewal 
patticiparts pas ane aA 

OO self-identific ation Dotter 
Comments 


Step 5: Review sampling methods 


5-1 Sampling technique considered and specified the Oo met Partially met (Not me 
true (or estimated) frequency of occurrence of the . 
event, the confidence interval to beused, amd the © not applicable © unable to determine 
margm of the error that willbe acceptable. 


Previow findings from: 
DO literature review Ci baseline assessm ent of indioss OJ otter 


Comments 
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5-2 The MCO/PIMP employed ena ge tier [Ey Weer re EE] tat mes 
techniques that protected against bias. fo Fon) qn ylncative mt neh te determine 
The type ofsam pling used: 
D)Probabilinr CD Nonprobability CO Random Osimple C1 stratified 
Oi convenience Ci suigment Oowta O cluster 

Comanents 
5-3 Sample contamed sufficientnumber of enrollees. fer [Ey terrier [Ey Pt re 

oO Neat enpiean oO UO \netiae te cbeernine 
Mof enrolkes in sam plirg frame Nofsamfple 
N of participants (ié., return rate) 
Cinnunents 


Step & Revievedata callection pracedii‘es 


6.1 Study design clearly sp ecified the datato be Let tides Mares 
collected. 
@ FeV ON her | Ci ves deere etveeniadate 


Conmuments 
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Performance Management Solutions Group 


6.2 The study design clearly specified the sources fa 2 | ee i 
of data. 
fem einen fi! TU eo 
Source ofdata: 
(Jtremter ([JClaims Litroyder (Jotter 
Commas 
6.3 The study design specified a systematic method Eyes [Ey eer eee ea 
of collecting valid and reliable data thatrepresents 
the entire population to which the study's | hod eben! [Ey bette tr deere 
indicators apply. 
Conneant 
6.4 The mstum ents for data collection pro vided for fe i fe hose 
consistent, accurate data collection over the time 
perio ds studied. fl! vA Siow fl! Here heyteremnnn 
Instrum ent(s) wed 
(survey (Medical Record Abstraction Tool (J other 
)) ONO 
® 2005 Behavioral Healthy Oovrcepts, Inc. Paue Gor |} 
77 


Inc. 


MO HealthNet Managed Care External Quality Review Appendix 2 


Supplemental Report — 2013 Performance Improvement Project (PIP) Worksheets 
6.5 The study design prospectively specified adata fe [Ey Prteew ee fa Wr mes 
analysis plan. : 

Oo Nol arpaeaiie Bi! i do el erriie 
Coma ees 
6.6 Qualified staff and personnal were used to collect mM [iE] taney met [i Nocme 
thedaa. 


o Poet any here ee | Hua Né te detemare 


Name Tk 


Role(s) of Project Leader 





Conmurerts 


Step 7: Assess impravement strategies 


7-4 Reasonableinterventions were undertaken to By ve [Ey ervey ves [Ey fers res 

address causes barriers identified through data 

amalysis and QI processes undertaken. By Nol anpticatie Ly nati co dsteanine 
Describe Irterverition: 

Comntants “ 7 
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Step 8: Review data analysis and interpretation of study results 





NAif study is not yet complete 


8.1 An analysis of thefindings was performed @ Met [Ey Partially met Ey Not met 
according to data analysis pla. OO] Not applicahke & Unabk to determine 


Not metifstudyis complete and no indication of a data analysis plan (see step 6.5) 


Comments 


$.2 The NCO /PIHP presented numerical PIP results 


r = Met 6 Partially met Go Not met 
and finding ‘accurately and clenrky. By] Not applicable [yy Unabke ta determine 
CO Ave tables ard figures labeled? (Ci Labekd clearly, accurately? 
comments 


8.3 The analysis identifiedinitial ndrepeat 


nut mdatird fj Me [iy Partially met [iy Net met 
measurements, statistical significance, factors that 

influence comparability of mitial and rep eat Go Nat applicable ia Unable to determine 
measurement, and factors that threaten internal and 

extern al validity. 


Indicate time periads af 
measurements: 


Indicate statistical analy ses 
used: 





Indicate statistical significance kvel ar 


confidence level used: HW 2% 9% [i] Unatie to deternine 





Comments 
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8.4 Analysis of study datainchuded an interpretation = met [Dy Partially met [not me 
of the extent to which its PIP was successful and 
follow-up activities. [Not applicable (J unable to determine 
Limitations described: 
Corelusions regarding the suocess 
of the interpretation: 


Recommendations for follow-up 
comment 








Step 9: Assess whether improvement is “real” improvement 
Note: NA only ifstudy period is not yet complete: otherwise " Unable to Determine”™ or “No” 

















9.1 The same methodology as the baselme Met fi Partially met =] Not met 
measurement was used when measurement was . . . 
repeated. fl Not applicahe [Rl Unable to determine 
Same source ofdata Gre OJ se By] Not applicable Bl Unable ta determine 
Same method ofdataollection [re Ne Bi] Not applicable By Unable ta determine 
Same participants examined Gres No By Not applicable By Unable to determine 
Same took wed Gre OJ se [ Not applicable Bl Unable to determine 
Comments 
9-2 Therewas a documented, quantitative Hy Met [i Partially met [Not met 
Improvement proces or outcomes of care . . : 
Bjx« appliahk Bt nable to determine 
(increased [decrease 
Statistical significaroe Clinical sigrificaruce 
Comments 
© 2005 Behavioral Health Concepts, Inc. Page Sot 11 
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9.3 The reported improvementin peformancehave ([]] Met [[] Partially met (] Not me 
"face" validity:i.e., the improvement performance 
app ears to be the result of the planned quality [4] Not applicable (unable to determire 
improvement intervention. 
Degree to which the intenention was ; ; 
coe ; (No rekvance Osmall DO Fair DO High 
Comments 
9.4 Thereis statistical evidence that any observed [Ey Met [EyPartialy met Nat met 
pet Seren ee eT ee Bo Nat applicable Oo Unable to determine 
J weak ( Moderate i stwng 
Comments 


Step 10: Assess sustained improvement 
CD Partially met (CD Not met 


10.1 Sustainedimprovement was demonstrated 0 Met 
through rep eated measurements over comparable [] Not applicatie 
time periods. 

comments 


Oo Unahk to determine 
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ACTIVITY 3: EVALUATE OVERALL VALIDITY AND RELIABILITY OF STUDY 
RESULTS: SUMMARY OF AGGREGATE VALIDATION FINDINGS AND 
RECOMMENDATIONS 

Conclusions 

Recommendations 
Check one: 
(High confidence is reported [Low confidence level is reported in MCO/PIHP PIP results 
( Moderate confidence is reported MCO/PIHP PIP results () Reported MCO/PIHP PIP results not credible 
(Not Applicable, study rot com plete 
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Appendix 3 — Performance Measures Request Documents 


Performance Measure Validation General Instructions 
Request Date: 1/22/2014 


Mail To: 

External Quality Review Submission 
Behavioral Health Concepts, Inc. 
4250 E. Broadway, Ste. 1055 
Columbia, MO 65201 


Priority Due Date: February 25, 2014 
FINAL Due Date: March 4, 2014 (due in BHC offices by 3pm) 


When applicable, submit one for each of the three measures: 
e Annual Dental Visit (ADV) 
e Childhood Immunization Status (CIS) 
e Follow-up After Hospitalization for Mental Illness (FUH) 


Unless otherwise indicated, please send all documents on CD or thumb drive using the “tab 
numbers” as titles for each document. If an item is not applicable or not available, please indicate 
this in a file on the CD that corresponds to that tab. 





Electronic Data Submission Instructions: 
(The file layouts to be used for each measure are detailed on pages 2-5 of this document.) 





e Make all submissions using compact disk or thumb drive formats (CD). Data files submitted via 
e-mail will not be reviewed. Insure that files on the CD are accessible on a Microsoft Windows 
7 workstation environment prior to submitting. 

e All files or CDs must be password protected. Do not write the password on the CD. Please 
email the password separately to amccurry@pmsginfo.com. Do not include the password 
anywhere on the CD, or in any correspondence sent with the CD. 

e Data file formats all need to be ASCII, and readable in a Microsoft Windows 7 
environment. Please be sure to name data columns with the same variable names that appear in 
the following data layout descriptions. 

Please include the column names as the first row of data in the file. 
All files must be @ delimited with no text qualifiers (i.e. no quotation marks around 
text fields). 

e Please ensure that date fields are in MM-DD-YYYY format and contain either a null value or a 
valid date. 

e For fields such as Enroll_Last where a member is still enrolled (and therefore a date has not yet 
been determined), the entry must be a valid future date (i.e. a value of | 2-/2-2300 would be 
acceptable to indicate current enrollment; a value of 12-12-1700 would not.) 

e Files will be accepted only in the specified layout. Please avoid adding extra columns or 
renaming the columns we have requested*. Files submitted in any other form will be 


rejected and not validated. 
*Note this especially in the FUH data file layout 
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There should be 3 separate data files submitted for each measure: 
File |. Enrollment Data 
File 2. Denominator and numerator file 
File 3. Sample selection (cases that were selected for medical record review; this file is 
submitted for Hybrid measures only) 





Please contact BHC prior to the submission deadline if you have any questions regarding these 
layouts or the data submission requirements, and we will be happy to assist you. 





All files received prior to/on the Priority Due Date will be reviewed by BHC personnel. 
Any glaring errors in data format, column format, etc will be noted and you will be 
allowed to resubmit a corrected file prior to the Final Due Date. After the Final Due 
Date, no new data files will be accepted. 
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Annual Dental Visit (ADV) 
(Administrative Only) 


File |. Enrollment Data 
Please provide all enrollment periods for each eligible Managed Care Member to verify 
continuous enrollment and enrollment gaps. 


Field Name Acceptable Content Description 
MCHP Any basic text and/or numbers Managed Care Health Plan name 
MEASURE ADV Annual Dental Visit 
The Missouri Medicaid recipient identification number (not 
DCN Whole numbers only the MCHPs internal tracking number) 
MEMBR_FIRST Any basic text Managed Care Member First Name 
MEMBR_LAST Any basic text Managed Care Member Last Name 
Numbers only in a correct date 
DOB format (ex. mm/dd/yyyy) Managed Care Member date of birth 
Numbers only in a correct date 
ENROLL_FIRST format (ex. mm/dd/yyyy) First date of enrollment 
Numbers only in a correct date 
ENROLL_LAST format (ex. mm/dd/yyyy) Last date of enrollment 


File 2. Denominator and Numerator Data 


Field Name Acceptable Content Description 
MCHP Any basic text and/or numbers Managed Care Health Plan name 
MEASURE ADV Annual Dental Visit 
The Missouri Medicaid recipient identification number 
DCN Whole numbers only (not the MCHPs internal tracking number) 
MEMBR_FIRST Any basic text Managed Care Member First Name 
MEMBR_LAST Any basic text Managed Care Member Last Name 
Numbers only in a correct date 
DOB format (ex. mm/dd/yyyy) Managed Care Member date of birth 
Numbers only in a correct date 
SER_DATE format (ex. mm/dd/yyyy) Date of service 
SER_CODE Any basic text and/or numbers Code used to identify numerator event 
Type of coding system: C=CPT Codes; H=HCPCS/CDT-3 
CODING_TYPE C, H, or | Codes*; |=I1CD-9-CM Codes. 
Administrative numerator event (positive case "hit"): 
ADMIN_HIT YorN y=yes; n=no 
EXCLUD YorN Was the case excluded from denominator Y=Yes; N=No 
EXCLUD_REASON | Any basic text and/or numbers Reason for exclusion 


* CDT is the equivalent dental version of the CPT physician procedural coding system. 
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Childhood Immunization Status (CIS) 


(Administrative or Hybrid) 


File |. Enrollment Data 


Please provide all enrollment periods for each eligible Managed Care Member to verify 


continuous enrollment and enrollment gaps. 


Field Name Acceptable Content 
MCHP Any basic text and/or numbers 
MEASURE cis 
DCN Whole numbers only 
MEMBR_FIRST Any basic text 
MEMBR_LAST Any basic text 

Numbers only in a correct date 
DOB format (ex. mm/dd/yyyy) 


Numbers only in a correct date 
format (ex. mm/dd/yyyy) 
Numbers only in a correct date 
format (ex. mm/dd/yyyy) 


ENROLL_FIRST 


ENROLL_LAST 


File 2. Denominator and Numerator Data 


Field Name Acceptable Content 
MCHP Any basic text and/or numbers 
Measure CIS 
DCN Whole numbers only 
MEMBR_FIRST Any basic text 
MEMBR_LAST Any basic text 
Numbers only in a correct date 
DOB format (ex. mm/dd/yyyy) 
Numbers only in a correct date 
SER_DATE format (ex. mm/dd/yyyy) 
SER_CODE Any basic text and/or numbers 
CODING_TYPE Corl 


Description 
Managed Care Health Plan name 


Childhood Immunization Status (Combo 3) 
The Missouri Medicaid recipient identification number 
(not the MCHPs internal tracking number) 


Managed Care Member First Name 
Managed Care Member Last Name 


Managed Care Member date of birth 
First date of enrollment 


Last date of enrollment 


Description 
Managed Care Health Plan name 


Childhood Immunization Status (Combo 3) 
The Missouri Medicaid recipient identification number 
(not the MCHPs internal tracking number) 


Managed Care Member First Name 
Managed Care Member Last Name 


Managed Care Member date of birth 


Date of service 


Code used to identify numerator event 
Type of coding system: C=CPT Codes; |=ICD-9-CM 
Codes 





ADMIN_HIT 
EXCLUD YorN 


EXCLUD_REASON Any basic text and/or numbers 


Administrative numerator event (positive case "hit"): 
y=yes; n=no 


Was the case excluded from denominator Y=Yes; N=No 


Reason for exclusion 
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Childhood Immunization Status (CIS) 
(Administrative or Hybrid) 


File 3. For Hybrid method ONLY - please provide a listing of the cases 
selected for medical record review. Use the following layout: 


Field Name Acceptable Content Description 
MCHP Any basic text and/or numbers MOHealthNet Managed Care Health Plan name 
MEASURE cis Childhood Immunization Status (Combo 3) 

The Missouri Medicaid recipient identification number (not 
DCN Whole numbers only the MCHPs internal tracking number) 
MEMBR_FIRST Any basic text Managed Care Member First Name 
MEMBR_LAST Any basic text Managed Care Member Last Name 

Numbers only in a correct date 

DOB format (ex. mm/dd/yyyy) Managed Care Member date of birth 

Medical record review status: 
MR_STATUS Ror NRors R = reviewed; NR = not reviewed; S = substituted 
PROVIDER_NAME | Any basic text and/or numbers Primary Care Provider who supplied the record 
PROVIDER_ID Any basic text and/or numbers Primary Care Provider identification number 
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Follow-Up After Hospitalization for Mental Illness (FUH) 
(Administrative Only) 


File |. Enrollment Data 
Please provide all enrollment periods for each eligible Managed Care Member to verify 
continuous enrollment and enrollment gaps. 


Field Name Acceptable Content Description 
MCHP Any basic text and/or numbers Managed Care Health Plan name 
MEASURE FUH Follow-Up After Hospitalization for Mental Illness 
The Missouri Medicaid recipient identification number (not 

DCN Whole numbers only the MCHPs internal tracking number) 
MEMBR_FIRST Any basic text Managed Care Member First Name 
MEMBR_LAST Any basic text Managed Care Member Last Name 

Numbers only in a correct date 
DOB format (ex. mm/dd/yyyy) Managed Care Member date of birth 

Numbers only in a correct date 
ENROLL_FIRST format (ex. mm/dd/yyyy) First date of enrollment 

Numbers only in a correct date 
ENROLL_LAST format (ex. mm/dd/yyyy) Last date of enrollment 


File 2. Denominator and Numerator Data 


Field Name Acceptable Content Description 
MCHP Any basic text and/or numbers Managed Care Organization name 
MEASURE FUH Follow-Up After Hospitalization for Mental IIlness 
The Missouri Medicaid recipient identification number 
DCN Whole numbers only (not the MCHPs internal tracking number) 
MEMBR_FIRST Any basic text Managed Care Member First Name 
MEMBR_LAST Any basic text Managed Care Member Last Name 
Numbers only in a correct date 
DOB format (ex. mm/dd/yyyy) Managed Care Member date of birth 
Numbers only in a correct date Date of discharge from hospitalization applicable to this 
DISCHG_DATE format (ex. mm/dd/yyyy) date of service 
Numbers only in a correct date 
SER_DATE format (ex. mm/dd/yyyy) Date of service 
SER_CODE Any basic text and/or numbers Code used to identify numerator event 
Type of coding system: C=CPT Codes; U=UB-92 
CODING_TYPE C, U, or H Revenue Codes; H=HCPCS Codes. 
Administrative numerator event (positive case "hit"): 
ADMIN_HIT YorN y=yes; n=no 
EXCLUD YorN Was the case excluded from denominator Y=Yes; N=No 


EXCLUD_REASON Any basic text and/or numbers Reason for exclusion 


Please see the Performance Measure Validation Submission Requirements and the Summary 
of Calculation Methods for Performance Measures. 
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Performance Measure Validation Submission Requirements 


Instructions: The following listing includes relevant source data for the EQR process. Please submit information on a CD. 
Each file on the CD should correspond to the tab number and description in the spreadsheet below. Within 
each CD file, include information specific for each of the three measures for the Managed Care population. 
Some items may not apply. For example, if you do not use a HEDIS vendor and perform measure 
calculations on site, then you may not have documentation of electronic record transmissions. These items 
apply to processes, personnel, procedures, databases and documentation relevant to how the MCHP 
complies with HEDIS measure calculation, submission and reporting. 


If you have any questions about this request, contact Amy McCurry Schwartz, EQRO Project Director, 
amccurry @pmsginfo.com. 





Check submitted Use this field to indicate whether you have submitted this information. If you are not submitting the particular 
information, please indicate “NA”. You may have submitted the content by other means either on the BAT or as part of 
some other documentation. If so, indicate “submitted”, and reference the document (see below). 

Name of Source Please write the name of the document you are submitting for the item. If you are submitting pages from a procedure 


Document manual, indicate so by writing "HEDIS submission manual, pages xx — xx." 

MCHP Comments Use this space to write out any concerns you may have or any clarification that addresses any issues or concerns you 
may have regarding either the items requested or what you submitted in the response. 

Reviewed By (BHC use) This space will be for BHC staff use. The purpose will be for tracking what is received and what is not received. It will 
not indicate whether the documents actually address the specific issue. 
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Tab 


HEDIS Performance Measure 


Check if 
Submitted 
or NA 


Name of Source 
Document 


MCHP Comments 


Reviewed 
by (BHC 
use) 





HEDIS 2013 Data Submission Tool (MO DHSS 2013 Table 
B HEDIS Data Submission Tool) for all three measures for 
the MOHealthNet Managed Care Population only. Do not 
include other measures or populations. 





HEDIS 2013 Audit Report. This is the HEDIS 
Performance Audit Report for the Managed Care Program 
product line and the three measures to be validated 
(complete report). If the three measures to be validated 
were not audited or if they were not audited for the 
Managed Care Program population, please send the 
report, as it contains Information Systems Capability 
Assessment information that can be used as part of the 
Protocol. 

RoadMap for HEDIS 2013. The information submitted for 
the RoadMap will include descriptions of the process for 
calculating measures for the MOHealthNet Managed Care 
Program population. 





List of cases for denominator with all HEDIS 2013 data 
elements specified in the measures. 











List of cases for numerators with all HEDIS 2013 data 
elements specified in the measures, including fields for 
claims data and MOHSAIC, or other administrative data 
used. Please note that one of the review elements in the 
Protocol is: The “MCO/PIHP has retained copies of files 
or databases used for performance measure reporting, in 
the event that results need to be reproduced.” 
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Tab 


HEDIS Performance Measure 


Check if 
Submitted 
or NA 


Name of Source 
Document 


MCHP Comments 


Reviewed 
by (BHC 
use) 





List of cases for which medical records were reviewed, 
with all HEDIS 2013 data elements specified in the 
measures. Based on a random sample, BHC will request 
MCHPs to gather a maximum of 30 records per measure 
and submit copies of the records requested to BHC. 





Sample medical record tools used if hybrid method(s) 
were utilized for HEDIS 2013 Childhood Immunization 
Status measures for the Managed Care Program 
population; and instructions for reviewers. 





All worksheets, memos, minutes, documentation, policies 
and communications within the MCHP and with HEDIS 
auditors regarding the calculation of the selected 
measures. (please limit this to 30 (two-sided) pages 
in this submission — all other information can be 
reviewed onsite, as required). 











Policies, procedures, data and information used to 
produce numerators and denominators. 
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Tab | HEDIS Performance Measure Check if | Name of Source MCHP Comments Reviewed 
Submitted | pocument by (BHC 
or NA use) 
10. | Policies, procedures, and data used to implement sampling 
(if sampling was used). At a minimum, this should include 
documentation to facilitate evaluation of: 
a. Statistical testing of results and any corrections 
or adjustments made after processing. 
b. Description of sampling techniques and 
documentation that assures the reviewer that 
samples used for baseline and repeat 
measurements of the performance measures 
were chosen using the same sampling frame 
and methodology. 
c. Documentation of calculation for changes in 
performance from previous periods (if 
comparisons were made), including tests of 
statistical significance. 
11. | Policies and procedures for mapping non-standard codes. 
12. | Record and file formats and descriptions for entry, 
intermediate, and repository files. 
13. | Electronic transmission procedures documentation. (This 





will apply if the MCHP sends or receives data 
electronically from vendors performing the HEDIS 
abstractions, calculations or data entry.) 
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Tab | HEDIS Performance Measure Check if | Name of Source MCHP Comments Reviewed 
Submitted | pocument by (BHC 
or NA use) 
14. Descriptive documentation for data entry, transfer, and 
manipulation of programs and processes. 

15. | Samples of data from repository and transaction files to 

assess accuracy and completeness of the transfer process. 
16. | Documentation of proper run controls and of staff review 

of report runs. 
17. | Documentation of results of statistical tests and any 

corrections or adjustments to data along with justification 

for such corrections or adjustments. 
18. | Documentation of sources of any supporting external data 

or prior years’ data used in reporting. 
19. | Procedures to identify, track, and link member enrollment 





by product line, product, geographic area, age, sex, 
member months, and member years. 
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Tab | HEDIS Performance Measure Check if | Name of Source MCHP Comments Reviewed 
Submitted | pocument by (BHC 
or NA use) 
20. | Procedures to track individual members through 
enrollment, disenrollment, and possible re-enrollment. 
21. | Procedures used to link member months to member age. 
22. | Documentation of “frozen” or archived files from which 
the samples were drawn, and if applicable, documentation 
of the MCHP’s/PIHP’s process to re-draw a sample or 
obtain necessary replacements. 
23. | Procedures to capture data that may reside outside the 
MCO’s/PIHP’s data sets (e.g. MOHSAIC). 
24. | Policies, procedures, and materials that evidence proper 
training, supervision, and adequate tools for medical 
record abstraction tasks. (May include training material, 
checks of inter-rater reliability, etc.) 
25. | Information Systems Capabilities Assessment (ISCA) 





Appendix V 
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MO HealthNet Managed Care External Quality Review Appendix 3 
Report of Findings — 2013 Performance Measures (PM) Request Documents 


Performance Measures to be Calculated for Managed Care Members 
METHOD FOR CALCULATING HEDIS 2013 PERFORMANCE MEASURES 


Please complete this form and return via email to BHC. Please direct any questions to Amy McCurry 
Schwartz. 


MCHP 

Date Completed 
Contact Person 
Phone 


Fax 


NCQA Accredited for MOHealthNet Product 
(Yes/No) 


Certified HEDIS Software Vendor and Software 
Record Abstraction Vendor 


What was the reporting Date for HEDIS 2013 
Measures? 


What was the Audit Designation (Report/No 
Report/Not Applicable)? 


Was the measure publicly Reported (Yes/No)? 
Did denominator include members who 
switched MCHPs (Yes/No)? 

Did denominator include members who 
switched product lines (Yes/No)? 


Did the denominator include 1115 Waiver 
Members (Yes/No)? 


Were proprietary or other codes (HCPC, NDC) 
used? 


Were exclusions calculated (Yes/No)? 


On what date was the sample drawn? 
Were exclusions calculated (Yes/No)? 
How many medical records were requested? 


How many medical records were received? 


How many medical records were substituted 
due to errors in sampling? 


How many medical records were substituted 
due to exclusions being measured? 
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Appendix 4 - Performance Improvement Project Request Documents 


Behavioral Health Concepts, Inc. 


4250 East Broadway, Suite 1055, Columbia, MO 65201 (573) 446-0405 
(573) 446-1816 (fax) 
(866) 463-6242 (toll-free) 
www.bhcinfo.com 


February 26, 2014 


Re: 2013 External Quality Review of the MO HealthNet Managed Care Program 
Performance Improvement Project Submission Request 


Dear 


This letter represents a request for information for the 2013 External Quality Review of 
MO HealthNet Health Plans, conducted by Behavioral Health Concepts, Inc., (BHC). 
With this correspondence we are requesting submission of all information pertaining to 
the Performance Improvement Projects (PIP) selected for validation for 2013. The topics 
chosen for <MCHP?> include: 


The due date for submission of this information is March 25, 2014. Please send all information to 
BHC, 4250 East Broadway, Suite 1055, Columbia, MO 65201. 


The requested information should include relevant source data for the EQR process. If 
submitting printed versions, include printouts or copies of all required information. Submit 
information for each PIP to be validated for your Health Plan. You may mark PIP sections. 
Provide separate and distinct information for each PIP. We have included face sheets indicating 
the selected PIPs for your health plan. It is acceptable to submit this information electronically. 


Specific information about the implementation of the protocols can be found in the 
documents previously forwarded to all Health Plans for the EQRO orientation and in the 
corresponding CMS 2013 Protocols for External Quality Review. We look forward to 
working with you to implement the External Quality Review. 


Sincerely, 


Mona Prater, MPA 
EQRO Assistant Project Director 


cc: Andrea Smith, MO HealthNet 
Amy McCurry Schwartz, Project Director, BHC 
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Appendix 5 — Performance Measures Worksheets 


Final Performance Measure Validation Worksheet: HEDIS 2013 
Follow-up After Hospitalization for Mental Illness 


The percentage of discharges for members 6 years of age and older who 
were hospitalized for treatment of selected mental health disorders and who 
were seen on an outpatient basis or were in intermediate treatment with a 
mental health provider. 


Element Specifications Rating Comments 

Documentation 
Appropriate and complete measurement plans 
and programming specifications exist that 
include data sources, programming logic, and 
computer source code. 


Eligible Population 
6 years and older as of date of 
Age discharge. 
Date of discharge through 30 
Enrollment days. 








Gap No gaps in enrollment. 
Anchor date None. 
Medical and mental health 
Benefit (inpatient and outpatient) 
Discharged from an inpatient 
setting of an acute care facility 
(including acute care 
psychiatric facilities) with a 
discharge date occurring on or 
before December 1 of the 
measurement year and a 
principal |CD-9-CM diagnosis 
code indicating a mental health 
disorder specified in Table FUH- 
A. The MCHP should not count 
discharges from nonacute care 
facilities (e.g., residential care 
Event/diagnosis | or rehabilitation stays). 
Sampling 

















Sampling was unbiased. 





Sample treated all measures independently. 





Sample size and replacement methods met 
specifications. 
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Numerator 
Data sources used to calculate the numerator 
(e.g., member ID, claims files, medical records, 
provider files, pharmacy records, including those 
for members who received the services outside 
the MCHPs network) are complete and accurate. 
Calculation of the performance measure adhered 
to the specification for all components of the 
numerator of the performance measure. 
Documentation tools used were adequate. 











Integration of administrative and medical record 
data was adequate. 





The results of the medical record review 
validation substantiate the reported numerator. 


Denominator 


Data sources used to calculate the denominator 

(e.g., Claims files, medical records, provider files, 

oharmacy records) were complete and accurate. 
Reporting 


State specifications for reporting performance 
measures were followed. 


Estimate of Bias 


0 - 5 percentage points 
dorinas the > 5 - 10 percentage points 
impact of data > 10 - 20 percentage points 
incompleteness |_> 20 - 40 percentage points 
for this > 40 percentage points 
measure? Unable to determine 





What range 


























What is the Underreporting 
direction of the 
bias? Overreporting 
Audit Rating 
Fully Compliant = Measure was fully compliant with State specifications. 


Substantially Compliant = Measure was substantially compliant with State specifications and 
had only minor deviations that did not significantly bias the reported rate. 

Not Valid = Measure deviated from State specification such that the reported rate was 
significantly biased. This designation is also assigned to measures for which no rate was 
reported, although reporting of the rate was required. 

Not Applicable = No Members qualified 

Note: 2 = Met; 0 = Not Met 
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Final Performance Measure Validation Worksheet: HEDIS 2013 
Childhood Immunizations Status 
The percentage of children 2 years of age who had four diphtheria, tetanus and 
acellular pertussis (DTaP); three polio (IPV); one measles, mumps and rubella 
(MMR); three H influenza type B (HiB); three hepatitis B (HepB), one chicken pox 
(VZV); four pneumococcal conjugate (PCV); two hepatitis A (HepA); two or three 
rotavirus (RV); and two influenza (flu) vaccines by their second birthday. The 
measure calculates a rate for each vaccine & nine separate combination rates 


e e ot= s e KX « e eC 

Dre = s e 
Appropriate and complete measurement plans and 
programming specifications exist that include data 
sources, programming logic, and computer source 
code. 


Children who turn 2 years of age 
Age during the measurement year 





12 months prior to the child’s 
Enrollment second birthday 





No more than one gap in 
enrollment of up to 45 days during 
the 12 months prior to the child’s 
second birthday. To determine 
continuous enrollment for a 
Medicaid beneficiary for whom 
enrollment is verified monthly, the 
member may not have more than a 
1-month gap in coverage (i.e., a 
member whose coverage lapses for 
2 months [60 days] is not 

Gap continuously enrolled). 





Enrolled on the child’s second 
Anchor date birthday 





Benefit Medical 








Event/diagnosis | None 





Sampling was unbiased. 





Sample treated all measures independently. 





Sample size and replacement methods met 
specifications. 

= ‘. e 
Data sources used to calculate the numerator (e.g., 
member ID, claims files, medical records, provider 
files, pharmacy records, including those for members 
who received the services outside the MCHPs 
network) are complete and accurate. 
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Calculation of the performance measure adhered to 
the specification for all components of the numerator 
of the performance measure. 

Documentation tools used were adequate. 
Integration of administrative and medical record data 
was adequate. 

The results of the medical record review validation 
substantiate the reported numerator. 




















Denominator 


Data sources used to calculate the denominator (e.g., 
claims files, medical records, provider files, pharmacy 
records) were complete and accurate. 


Reporting | 
State specifications for reporting performance 
measures were followed. 
Estimate of Bias 
0 - 5 percentage points 
What range > 5 - 10 percentage points 
as ee ; > 10 - 20 percentage points 
Nsieleletsetins > 20 - 40 percentage points 
incompleteness ; 
for this > 40 percentage points 
measure? Unable to determine 





























What is the Underreporting 
direction of the 
bias? Overreporting 


Audit Rating 


Fully Compliant = Measure was fully compliant with State specifications. 
Substantially Compliant = Measure was substantially compliant with State 
specifications and had only minor deviations that did not significantly bias the 
reported rate. 

Not Valid = Measure deviated from State specification such that the reported rate 
was significantly biased. This designation is also assigned to measures for which no 
rate was reported, although reporting of the rate was required. 


Not Applicable = No Members qualified 
Note: 2 = Met; 0 = Not Met 
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Final Performance Measure Validation Worksheet: HEDIS 2013 
Annual Dental Visit 
The percentage of enrolled Managed Care Program Members who were 2 -21 years 
of age who had at least one dental visit during the measurement year. This 
measure applies only if dental care is a covered benefit in the MCHP's Medicaid 
contract. 


Element Specifications Rating Comments 
Documentation 
Appropriate and complete measurement plans and 
programming specifications exist that include data 
sources, programming logic, and computer source 
code. 
Eligible Population 


2 -21 years of age as of 
December 31, 2010. The 
measure is reported for 
each of the following age 
stratifications and as a 
combined rate: 

* 2 -3 year-olds 

* 4 -6 year-olds 

* 7-10 year-olds 

* 11 - 14 year-olds 

* 15 - 18 year-olds 

* 19 - 21 year-olds 


Enrollment Continuous during 2010 

No more than one gap in 
enrollment of up to 45 days 
during 2010. To determine 
continuous enrollment for a 
member for whom 
enrollment is verified 
monthly, the member may 
not have more than a 1- 
Gap month gap in coverage. 
Enrolled as of December 31, 
Anchor date 2010 


Benefit Medical 
Event/ diagnosis None 
Sampling - Not Applicable to this measure, calculated via 
Administrative calculation methodology only 


Age 
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Data sources used to calculate the numerator (e.g., 
member ID, claims files, medical records, provider 
files, pharmacy records, including those for 
members who received the services outside the 
MCHPs network) are complete and accurate. 





Calculation of the performance measure adhered to 
the specification for all components of the numerator 
of the performance measure. 


Data sources used to calculate the denominator 
(e.g., claims files, medical records, provider files, 
harmacy records) were complete and accurate. 





State specifications for reporting performance 
measures were followed. 














0 - 5 percentage points 





> 5 - 10 percentage points 








What defi > 10 - 20 percentage points 
at range defines ; 
the impact of data > 20 - 40 percentage points 








incompleteness for > 40 percentage points 
this measure? Unable to determine 


Underreporting 





What is the direction 
of the bias? Overreportin 














* « « 


Fully Compliant = Measure was fully compliant with State specifications. 
Substantially Compliant = Measure was substantially compliant with State 
specifications and had only minor deviations that did not significantly bias the 
reported rate. 

Not Valid = Measure deviated from State specification such that the reported rate 
was Significantly biased. This designation is also assigned to measures for which no 
rate was reported, although reporting of the rate was required. 

Not Applicable = No Members qualified 


Note: 2 = Met; 0 = Not Met 
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Appendix 6 — Performance Measures Medical Record Request Letter 
Behavioral Health Concepts, Inc. 


4250 E. Broadway, Suite 1055, Columbia, MO 65201 (573) 446-0405 
(573) 446-1816 (fax) 
(866) 463-6242 (toll-free) 
www.bhcinfo.com 


® 
March 25, 2014 





Subject: 2013 External Quality Review Performance Measure Validation 
Protocol Medical Records Request (hybrid methodology only). 


Due Date: April 29, 2014 by 3:00pm 





BHC has reviewed <MCHP’s> Childhood Immunization Status (CIS) Measure. 


Please find attached a file containing a listing of the cases related to this HEDIS Measure 
that have been selected for medical record review. Behavioral Health Concepts, Inc. 
(BHC) requests copies of all medical records for these sampled cases. Each medical 
record supplied should contain all the information that contributed to the numerator 
for the given HEDIS 2013 Measure. Please forward copies of these medical records to 
BHC at the following address and mark the package as confidential. 


Behavioral Health Concepts, Inc. 
Attn: Amy McCurry Schwartz 
4250 E. Broadway, Suite 1055 
Columbia, MO 65201 


If you have any questions, please contact BHC’s External Quality Review team at (573) 
446-0405 or via e-mail: amccurry@bhcinfo.com 





Thank you, 


Amy McCurry Schwartz 
EQRO Project Director 


Attachment: 
1) File containing a sample of cases for medical record review 


cc: Ms. Susan Eggen, Assistant Deputy Director, MO HealthNet Division, Missouri 
Department of Social Services 


Performance Management Solutions Group 103 
A division of Behavioral Health Concepts, Inc. 





MO HealthNet Managed Care External Quality Review Appendix 7 
Supplemental Report — 2013 Table of Contents for Medical Record Training Manual 


Appendix 7 - Table of Contents for Medical Record Training Manual 


Table of Contents 

Table of Contents 2 

Background of Project 3 

External Quality Review of Medicaid Managed Care 3 
Qualifications of Reviewers 3 
Confidentiality and Privacy 3 

Conflict of Interest 4 

Record Review Protocols 5 

Purpose of Medical Record Reviews 5 
Process of Request of Medical Records 5 
General Medical Record Review Guidelines 5 
Definition of Medical Record 5 

Claim Form or Claim History6 

Date Specificity 6 

Organization of Medical Records 6 
Childhood Immunization Status Protocol 6 
Background 6 

Time Period Reviewed 7 

Instructions 7 


CIS Abstraction Tool 8 


Requests for Medical Records 16 
Sample Medical Records 18 
Sample Claim Forms/Histories 20 
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Appendix 8 —- Performance Measures Medical Record Abstraction Tool 


Childhood Immunization Abstraction Tool 





Patient Name 





















































Date of Birth: 
Missing = 99999999 








Provider Name 


















































Name of MCO HealthCare USA (1) 


(Check only one) Home State (2) 
Missouri Care 
(3) 





Abstractor Initials 





























Date of abstraction 








Data entry operator 
initials 





























Start Time 
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Search the medical record for the complete immunization history 












































































































































































































































Source of 
Documentation: UL) Medical Record (1) 
Check One Q) Claim Form (2) 
QO) Both (3) 
L) None (0) 
Type of Documentation () = Dated Immunization History (1) 
Check One QL) Immunization Certificate (2) 
QO) Both (3) 
L) None (0) 
DTaP Date 1 m m d d y y y y 
Missing = 99999999 
Not Applicable = 88888888 
DTaP Date 2 m m d d y y y y 
Missing = 99999999 
Not Applicable = 88888888 
DTaP Date 3 m m d d y y y y 
Missing = 99999999 
Not Applicable = 88888888 
DTaP Date 4 m m d d y y y y 
Missing = 99999999 
Not Applicable = 88888888 
First Birthday m m d d y y y y 
42 days old m m d d y y y y 
Second Birthday m m d d y y y y 
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Notes: 


Source of 
Documentation: 


Check One 


Type of Documentation 
Check One 


IPV Date 1 


Missing = 99999999 
Not Applicable = 88888888 


IPV Date 2 


Missing = 99999999 
Not Applicable = 88888888 


IPV Date 3 


Missing = 99999999 
Not Applicable = 88888888 


First Birthday 


42 days old 


Appendix 8 
Abstraction Tools 







































































































































































Were any of the DTaP vaccines administered prior to the child's 42nd day of birth? QL) Yes (1) 
L) No (0) 
IPV 
LJ Medical Record (1) 
Q)) Claim Form (2) 
QO) Both (3) 
L) None (0) 
() Dated Immunization History (1) 
Q) = |mmunization Certificate (2) 
QO) Both (3) 
L) None (0) 
m m d d y y 
m m d d y y y 
m m d d y y y 
m m d d y y y 
m m d d y y y 
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Second Birthday m m d d y y y y 
Were any of thelPV vaccines administered prior to the child's 42nd day of birth? QO) Yes (1) 

LU No (0) 


Notes: 


Source of 



































Documentation: L) Medical Record (1) 
Check One Q)) Claim Form (2) 
Q) Both (3) 
L) None (0) 
Type of Documentation L) = Dated Immunization History (1) 
Check One Q) Immunization Certificate (2) 
Q) Both (3) 
L) None (0) 
Is There Evidence of a History of: 
Measles QO) Yes (1) 
OI No (0) 
Mumps QO) Yes (1) 
OI No (0) 
Rubella LU) Yes (1) 
LI No (0) 
Measles Seropositive Test Date m m d d y y y y 
Missing = 99999999 
Not Applicable = 88888888 
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Mumps Seropositive Test Date m m d d y y y y 





Missing = 99999999 
Not Applicable = 88888888 
































Rubella Seropositive Test Date m m d d y y y y 





Missing = 99999999 
Not Applicable = 88888888 
































MMR Date 1 m m d d y y y y 


Missing = 99999999 
































Not Applicable = 88888888 





Notes: 


Source of 




































































Documentation: L) Medical Record (1) 
Check One Q)) Claim Form (2) 

Q) Both (3) 

L) None (0) 
Type of Documentation L) Dated Immunization History (1) 
Check One LY = |mmunization Certificate (2) 

Q) Both (3) 

L} None (0) 
HiB Date 1 m m d d y y y y 
Missing = 99999999 
Not Applicable = 88888888 
HiB Date 2 m m d d y y y y 
Missing = 99999999 
Not Applicable = 88888888 
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HiB Date 3 m m d d y y y y 
Missing = 99999999 
Not Applicable = 88888888 






































































































































First Birthday m m d d y y y y 

42 days old m m d d y y y y 

Second Birthday m m d d y y y y 

Were any of the HiB vaccines administered prior to the child's 42nd day of birth? QU) Yes (1) 
L) No (0) 

Notes: 


Source of 


Documentation: Medical Record (1) 


(Check all that apply) Claim Form (2) 
Type of 


Documentation: Dated Immunization History (1) 


OO oo 


(Check only one) Immunization Certificate (2) 


Is there documented evidence of a history of Hep 
B? UL) Yes (1) 


L) No (0) 
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Hep B Seropositive Test Result Date m m d d y y y y 
Missing = 99999999 
Not Applicable = 88888888 



































Hep B Date 1 m m d d y y y y 





Missing = 99999999 
Not Applicable = 88888888 
At delivery/birth = 11111111 
































Hep B Date 2 m m d d y y y y 
Missing = 99999999 
Not Applicable = 88888888 



































Hep B Date 3 m m d d y y y y 
Missing = 99999999 
































Not Applicable = 88888888 





Notes: 


VZV 


Source of 


Documentation: L) Medical Record (1) 
Check One Q)) Claim Form (2) 

Q) Both (3) 

L) None (0) 
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Type of Documentation LI Dated Immunization History (1) 
Check One Q) = |mmunization Certificate (2) 

Q) Both (3) 

L) None (0) 
Is There Documented Evidence of a History of Chicken Pox? Q) Yes (1) 

L No (0) 
Date of positive Chicken Pox? m m d d y y y y 
Missing = 99999999 
Not Applicable = 88888888 
VZV Seropositive Test Result Date m m d d y y y y 
Missing = 99999999 
Not Applicable = 88888888 
VZV Date 1 m m d d y y y y 
Missing = 99999999 
Not Applicable = 88888888 
Notes: 
PCV 

Source of 
Documentation: L) Medical Record (1) 
Check One Q) Claim Form (2) 

UL) Both (3) 

L) None (0) 
Type of Documentation L) Dated Immunization History (1) 
Check One QL) Immunization Certificate (2) 

Q) Both (3) 

L) None (0) 
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PCV Date 1 m m d d y y y y 
Missing = 99999999 
Not Applicable = 88888888 



































PCV Date 2 m m d d y y y y 
Missing = 99999999 
Not Applicable = 88888888 



































PCV Date 3 m m d d y y y y 
Missing = 99999999 
Not Applicable = 88888888 



































PCV Date 4 m m d d y y y y 
Missing = 99999999 
Not Applicable = 88888888 









































































































































First Birthday m m d d y y y y 

42 days old m m d d y y y y 

Second Birthday m m d d y y y y 

Were any of the PCV vaccines administered prior to the child's 42nd day of birth? UL) Yes (1) 
L) No (0) 

Notes: 

h h m m 
End Time 
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Appendix 9 — Agenda for Site Visits 


Missouri HealthNet 
Managed Care Program 


External Quality 
TY 





SITE VISIT AGENDA 


Date Here — (Morning OR Afternoon 


TIME ACTIVITIY ATTENDEES LOCATION 


1:00 — 4:30 Case Management Document Mona Prater Conference Room — 
Review Lisa Heying Quiet Location 


1:00 — 1:30 Validation of Performance Amy McCurry 
Measures Schwartz 






Health Plan 
Attendees 


1:30 -— 4:30 Compliance Document Review Amy McCurry 
- Including Grievance Record Schwartz 
Review 








Date Here — Morning & Afternoon 

8:30 — 9:00 Introduction -- Opening BHC, Inc. — 
Amy McCurry 
Schwartz 
Mona Prater 
Lisa Heying 


Health Plan Attendees 


9:00-11:00 | Case Management & Compliance — | BHC, Inc. — 
Interviews Case Management Amy McCurry 
Staff Schwartz 
Mona Prater 
Lisa Heying 


Health Plan Attendees 


Performance Management Solutions Group 
A division of Behavioral Health Concepts, Inc. 


MO HealthNet Managed Care External Quality Review Appendix 9 
Suplemental Report — 2013 Agenda for Site Visits 


a 
/11:00- 11:30 |LunchBreak | 
11:30-1:30 | Case Management & Compliance BHC, Inc. — 
Review — Interviews with Amy McCurry 
Administrative Staff Schwartz 
Mona Prater 
Lisa Heying 


Health Plan Attendees 


1:45 — 3:00 Validation of Performance 
Improvement Projects Amy McCurry 
Schwartz 
Mona Prater 
Lisa Heying 


Health Plan Attendees 
| 3:00-3:15 | Exit Conference Preparation —_—| BHC, Inc. Staff. | 
3:15 - 4:00 Exit Conference BHC, Inc. — 
Amy McCurry 
Schwartz 
Mona Prater 
Lisa Heying 


Health Plan Attendees 
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Appendix 10 — Site Visit Information Request Letter 


Behavioral Health Concepts, Inc. 
4250 E. Broadway, Suite 1055, Columbia, MO 65201 (573) 446-0405 
(573) 446-1816 (fax) 
(866) 463-6242 (toll-free) 
www.bhcinfo.com 





® 


June 10, 2014 


RE: SITE VISIT AGENDA AND DOCUMENT REVIEW 
Dear Plan Administrator: 


We are finalizing plans for the on-site review of each Health Plan. The following 
information is being provided in an effort to make preparations for the on-site 
review as efficient as possible for you and your staff. The following information 
or persons will be needed at the time of the on-site review at <MCHP>. 


Performance Improvement Projects 


Time is scheduled in the afternoon to conduct follow-up questions, review data 
submitted, and provide verbal feedback to the Health Plan regarding the 
planning, implementation, and credibility of findings from the Performance 
Improvement Projects (PIPs). Any staff responsible for planning, conducting, 
and interpreting the findings of PIPs should be present during this time. The 
review will be limited to the projects and findings submitted for 2013. Please be 
prepared to provide and discuss any new data or additional information not 
originally submitted. Updated PIP information, with current data provided at the 
on-site review will be accepted and considered in the final audit assessment. 


Performance Measure Validation 


As you know, BHC is in the process of validating the following three performance 
measures: 


e HEDIS 2013 Annual Dental Visit (ADV) 
e HEDIS 2013 Childhood Immunization Status, Combo 3 (CIS) 
e HEDIS 2013 Follow-Up After Hospitalization for Mental Illness (FUH) 


BHC is following the CMS protocol for validating performance measures. The 
goals for this process are to: 


« Evaluate the accuracy the of Medicaid performance measures reported by 
the Health Plan; and 

«Determine the extent to which Medicaid-specific performance measures 
calculated by the Health Plan followed specifications established by the MO 
HealthNet Division. These specifications consist of the HEDIS 2013 
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Technical Specifications. 


To complete this process we will review the following documents while on-site: 


« Data Integration and Processes Used to Calculate and Report 
Performance Measures 


1. Documentation of the performance measure generating process 

2. Report production logs and run controls 

3. Documentation of computer queries, programming logic, or source code (if 
available) used to create denominators, numerators and interim data files - 
for each of the three measures 

4. Code mapping documentation 

5. Documentation of results of statistical tests and any corrections with 
justification for such changes, if applicable - for each of the three measures 

6. Documentation showing confidence intervals of calculations when sampling 
methodology used — for each of the three measures 

7. Description of the software specifications or programming languages 
instructions used to query each database to identify the denominator, and/or 
software manual 

8. Source code for identifying the eligible population and continuous enrollment 
calculation — for each of the three measures 

9. Description of the software specification or programming languages used to 
identify the numerator 

10. Programming logic and/or source code for arithmetic calculation of each 
measure to ensure adequate matching and linkage among different types of 
data 


« Sampling Validation 


1. Description of software used to execute sampling sort of population files 

2. Source code for how samples for hybrid measures were calculated 

3. Policies to maintain files from which the samples are drawn in order to keep 
population intact in the event that a sample must be re-drawn or 
replacements made 

4. Documentation that the computer source code or logic matches the 
specifications set forth for each performance measure, including sample size 
and exclusion methodology 

5. Documentation of “frozen” or archived files from which the samples were 
drawn 

6. Documentation assuring that sampling methodology treats all measures 
independently, and there is no correlation between drawn samples 


Performance Measure Interviews 


In addition to the documentation reviews, interviews will be conducted with the 

person(s) responsible for: 

= Overseeing the process of identifying eligible members from Health Plan data 
sources for the measures to be validated; 
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=" Programming the extraction of required elements from the Health Plan data 
sources for the measures to be validated; 

= Integrity checks and processes of verifying the accuracy of data elements for 
the measures to be validated; 

= Overseeing the process of medical record abstraction, training, and data 
collection for the measures to be validated; and 

=" Contractor oversight and management of any of the above activities. 


On-site activities may also include, but are not limited to, the following: 

= Demonstration of HEDIS software 

= Demonstration of the process for extracting data from Health Plan databases 
=" Possible data runs for identifying numerator and denominator cases 


Compliance & Case Management Project Review 


The final activity to prepare for during the on-site visit will be the compliance and 
case management review. Documentation review and interviews with MO 
HealthNet Division staff have occurred prior to the on-site visit. This will enable 
BHC to use the time at the Health Plan as efficiently as possible. The following 
information will be needed at the time of the on-site review: 


Compliance Documents 


=» Member Handbook 

* 2013 Marketing Plan and materials 

» 2013 Quality Improvement Committee minutes 
« Approved Case Management Policy 
Compliance 


Interviews with health plan compliance staff will be conducted as needed. 
Case Management Interviews 


The attached agenda requests an interview in the morning with case 
management staff. These interviews are focused on staff members who interact 
directly with members, and who provide case management or disease 
management services. 


We are asking that the case managers listed be available for the interviews. 
Additional case management staff is welcome to participate, as interview 
questions will include general questions regarding practices at the Health Plan. 


(names of case managers here) 


In some circumstances it may be necessary to conduct these interviews by 
telephone. In these instances, we request that speaker-phone equipment be 
available in the conference room being utilized by the review team. Please 
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ensure that the requested staff is available in their location at the identified 
interview time. 


Interviews in the late morning are scheduled to include administrative staff. It 
would be helpful to include the following staff: 


Plan Director 

Medical Director 

Quality Assurance Director 

Case Management Supervisors or Administrators 
Utilization Management Director 


This year we have attempted to eliminate concurrent activities and interviews 
during the full on-site review date. These interviews, including required 
telephone interviews can be scheduled in a convenient location in your offices. 
On the day that document reviews are scheduled for the compliance & case 
management review, a separate conference room or meeting space will be 
needed to conduct the performance measure interviews and document review. 
Also, the on-site review team will need to order a working lunch on the full day 
visit. If lunch facilities are not available, please provide the name and telephone 
number of a service in your vicinity that can accommodate ordering lunch. Your 
assistance will be appreciated. 


The Health Plan staff involved in any of the referenced interviews or activities, or 
anyone identified by the Health Plan, is welcome to attend the introduction and/or 
the exit interview. 


Again, your assistance in organizing the documents, individuals to be 
interviewed, and the day’s activities is appreciated. If you have questions, or 
need additional information, please let me know. 


Sincerely, 


Mona Prater 
Assistant Project Director 


Cc: Amy McCurry Schwartz, Esq., Project Director 
Susan Eggen, MO HealthNet Division 
Andrea Smith, MO HealthNet Division 
Myrna Bruning, Consultant 


Attachment: 
On-Site Review Agenda 


Performance Management Solutions Group 119 
A division of Behavioral Health Concepts, Inc. 





MO HealthNet Managed Care External Quality Review Appendix || 
Supplemental Report — 2013 Compliance Review Scoring Form 


Appendix | | — Compliance Review Scoring Form 


2013 BHC MCHP Compliance Review Scoring Form 


This document is used to score the number of items met for each regulation 
by the MCHP. 


1. Review all available documents prior to the site visit. 


2. Follow-up on incomplete items during the site visit. 
3. Use this form and the findings of Interviews and all completed protocols 
to complete the Documentation and Reporting Tool and rate the extent to 
which each regulation is met, partially met, or not met. 
Scores from this form will be used to compare document compliance across 
all MCHPs. 
0 = Not Met: Compliance with federal regulations could not be 
validated. 
1 = Partially Met: MCHP practice or documentation indicating 
compliance was observed, but total compliance could not be validated. 
2 = Met: Documentation is complete, and on-site review produced 
evidence that MCHP practice met the standard of compliance with 
federal regulations. 
























































2010 2009 
Rating Rating 
0=Not | 0=Not 
2011 Met Met 
Site = = 
Contract Visit | Partially | Partially 
Compliance and Met Met 
Tool Federal Regulation | Description Comments | Findings | 2=Met | 2=Met 
Subpart C: Enrollee Rights and Protections 
2.6.1(a)1- 
25, 
2.2.6(a), Enrollee Rights: 
1 | 2.6.2(j) 438.100(a) General Rule 
2.6.1(a)1, 
2.9, 
2.6.2(j), Enrollee Rights: 
2 | 2.6.2(n) 438.10(b) Basic Rule 
Alternative 
Language: 
2.15.2(e), Prevalent 
3 | 2.8.2 438.10(c)(3) Languages 
Language and 
2.8.2, format: 
2.8.3, Interpreter 
4 | 2.6.2(n)(2) | 438.10(c)(4,5) Services 
Information 
Requirements: 
2.6.1(a)1, Alternative 
5 | 2.6.2(n)1 438.10(d)(1)(i) Formats 
2.6.1(a)1, 
2.6.2(n)2 - 
dot point Information 
35, Requirements: 
2.6.2(q), 438.10(d)(1)(ii)Jand | Easily 
6 | 2.8.2, 2.8.3 | (2) Understood 
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2.3.5, 
2.6.1(a)2/3 
, 2.6.2(k)1, 
2.6.2(n), Enrollee Rights: 
2.6.2(n)(2), Information, 
7 | 2.6.2(q) 438.10(f) Free Choice 
Information to 
Enrollees: 
Physician 
8 | 2.6.2(n)(2) | 438.10 (g) Incentive Plans 
2.4, 2.4.5, 
2.4.5(a)2- 
; Liability for 
2.20.1(all), Payment and 
9 | 3.5.3(f) 438.10(i) Cost Sharing 
2.2.6(a), 
2.2.6(b), Specific Enrollee 
2.6.1(a)(3), Rights: Provider- 
2.6.2(j), Enrollee 
10 | 2.9.1 438.100(b)(2)(iii) Communications 
Right to 
2.6.2(j), Services, 
2.30.1, including right of 
2.30.2, refusal. Advance 
11 | 2.30.3 438.100(b)(2)(iv,v) | Directives 
2.6.2(j), 
2.4.8, 2.13, Right to 
12 | 2.14 438.100(b)(3) Services 
2.2.6, 
2.14.3, Compliance with 
2.14.8, Other State 
13 | 2.14.9 438.100(d) Requirements 
Total Enrollee Rights and Protections 
Subpart D: Quality Assessment and Performance Improvement 
Subpart D: Quality Assessment and Performance Improvement: Access Siandards 
Dhol 
2.6.2(j), Availability of 
2.14.3, Services: 
2.7.1(g), Provider 
14 | 3.5.3 438.206(b)(1)(i-v) _| Network 
2.7.1(e), Access to Well 
2.7.1 (f), Woman Care: 
15 | 2.14.8 438.206(b)(2) Direct Access 
Second 
438.206(b)(3) Opinions 
Out of Network 
Services: 
Adequate and 
Timely 
438.206(b)(4) Coverage 
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2.14.5 
Out of Network 
2.4, Providers: Cost 
18 | 2.20.1(d) 438.206(b)(5) Sharing 
2.3.14(a)2, 
2.14.1, 
2.14.4(a-f), 
2.17.1, 
19 | 3.5.3 438.206(c)(1)(i-vi)_| Timely Access 
2.2.6(a)1- Cultural 
20 | 3, 2.17.1 438.206(c)(2) Considerations 
Primary Care 
and 
Coordination of 
2.14.11, Healthcare 
21 | 2.3.5(e) 438.208(b) Services 
2.6.2(m), Care 
2.14.11, Coordination: 
22 | 2.5.3(e) 438.208(c)(1) Identification 
2.12.10, 
2.14.2(c), 
2.14.11, 
2.17.5, 
Attachment 
3- 
Children 
with 
Special Care 
Healthcare Coordination: 
23 | Needs 438.208(c)(2) Assessment 
Care 
Pallas CoV, Coordination: 
24 | 2.14.11 438.208(c)(3) Treatment Plans 
2.3.8, 
2.3.7, 
2.6.1(k)(3), 
2.14.6, Access to 
25 | 2.14.7 438.208(c)(4) Specialists 
2.2.1(i), 
2 Sily 
2.7.4, 
2.9.2, 
2.10.2, 
2.14.1, 
2.14.2(a- 
h); 
2.14.2(d)1- Authorization of 
26 | 2 438.210(b) Services 
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2.15.4, Notice of 
27 | 2.14.2(d)6 | 438.210(c) Adverse Action 
2.6.2(k)(3), 
2.14.2(d)6, 
2.15.4(a- 
C), Timeframe for 
28 | 2.16.3(e) 438.210(d) Decisions 
Compensation 
for Utilization 
Management 
29 | 2.17.5(b) 438.210(e) Decisions 
2.4.8, 
lel, Emergency and 
2.7.1(y), Pos-stabilization 
2.7.3(V), pgs 24/25 Rev. 
30 | 2.14.2 438.114 Checklist 
Subpart D: Quality Assessment and Performance Improvement: Structure and 
Operation Siandards 
General Rules for 
2.17.2(n), Credentialing and 
31 | 2.17.5(c), 2.30.2 438.214(a,b) Recredentialing 
Nondiscrimination 
and Provider 
438.214(c) and Discrimination 
32 | 2.2.6(b)(c) 438.12 Prohibited 
Excluded 
33 | 2.31.5 438.214(d) Providers 
Other State 
Requirements: 
34 | 2.3.9, 2.3.17 438.214(e) Provider Selection 
2.6.2(n)(2), Disenrollment: 
2.6.2(s)(all), 438.226 and Requirements and 
35 | 2.6.2(u) 438.56(b)(1-3) Limitations 
Disenrollment 
2:1, 202, ZL), Requested by 
36 | 2.6.1(g), 2.6.2® 438.56(c) Enrollee 
Procedures for 
Disenrollment -- 
Pgs 29/30 Rev. 
37 | 2.6.2(r,s-1,t) 438.56(d) Checklist 
Timeframe for 
Disenrollment 
38 | 2.6.2(u) 438.56(e) Determinations 
Grievance 
39 | 2.15, 2.15.3(a,b) 438.228 Systems 
2.6.1(a)(18), 
2.16.2(c), Subcontractual 
2.31.2(a)8, 2.31.3, Relationships and 
40 | 3.5.1, 3.5.2, 3.5.3 | 438.230(a,b) Delegation 
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Subpart D: Quality Assessment and Performance Improvement: Measurement and 



























































Improvement 
There is 
very little in 
the contract 
compliance 
tool 
Adoption of regarding 
Practice practice 
41 | 2.17.2(d) 438.236(b)(1-4) Guidelines guidelines. 
Dissemination of 
Practice 
42 | 2.17.2(d) 438.236(c) Guidelines 
Application of 
Practice 
Guidelines -- Pgs 
32/33 of Rev. 
43 | 2.17.2(d,f) 438.236(d) Checklist 
Quality 
Assessment and 
Improvement 
44 | 2.17.1, 2.17.5 438.240(a)(1) Program 
438.240(b)(1) Basic Elements of 
45 | 2.17.5(d) and 438.240(d) MCO Ql and PIPs 
2.17, 2.17.3, 438.240(b)(2)(c) | Performance 
46 | Attachment 6 and 438.204(c) Measurement 
Basic elements of 
MCO Ql and 
PIPs: Monitoring 
47 | 2.17.5(b) 438.240(b)(3) Utilization 
Basic elements of 
48 | 2.17.5 438.240(b)(4) MCO QI and PIPs 
Attachment 6 - 
State Quality Program Review 
49 | Strategy 438.240(e) by State 
Health Information 
50 | 2.25 438.242(a) Systems 
2.25(all) - 2.25.1, 
2.25.2(a,b), Basic Elements of 
51 | 2.25.3, 2.25.4 438.242(b)(1,2) HIS 
Basic Elements of 
52 | 2.26.1, 2.29.1 438.242(b)(3) HIS 
Total Quality Improvement and 
Assessment 














Subpart F: Grievance Systems 
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Grievance and 
Appeals: General 


















































53 | 2.15 438.402(a) Requirements 
Grievance and 
2.15.2, 2.15.5(a), Appeals: Filing 
54 | 2.15.6(a) 438.402(b)(1) Authority 
Grievance and 
55 | 2.15.6(a) 438.402(b)(2) Appeals: Timing 
2.15.2(a), Grievance and 
2.15.5(a), Appeals: 
56 | 2.15.6(a,b) 438.402(b)(3) Procedures 
Notice of Action: 
2.15.2(e), Language and 
57 | 2.15.4(a),2.6.2(q) | 438.404(a) Format 
Notice of Action: 
58 | 2.15.4(b) 438.404(b) Content 
Notice of Action: 
59 | 2.15.4(c) 438.404(c) Timing 
Handling of 
Grievances and 
2.15.5(b,c,d), Appeals: General 
60 | 2.15.6(h,i,j) 438.406(a) Requirements 
Handling of 
Grievances and 
2.15.6(g) 2.15.6(h) Appeals: Special 
61 | 2.15.6(i) 2.15.6()) | 438.406(b) Requirements 
Resolution and 
notification: 
Grievances and 
2.15.5(e), Appeals - Basic 
62 | 2.15.6(k) 438.408(a) rule 
Resolution and 
notification: 
Grievances and 
Appeals - 
2.15.5(e,f), Timeframes and 
63 | 2.15.6(k-l) 438.408(b,c) extensions 
Resolution and 
notification: 
Grievances and 
2.15.5(e), Appeals - Format 
64 | 2.15.6(k,m) 438.408(d)(e) and content 
Resolution and 
notification: 
Grievances and 
Appeals - 
2.15.2(i), Requirements for 
65 | 2.15.6(m) 438.408(f) State fair hearing 
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Expedited 
resolution of 
66 | 2.15.6(n,o) 438.410 appeals 





Information about 
the grievance 





systems of 

providers and 
67 | 2.15.2(c), 3.5.3(c) | 438.414 subcontractors 

Recordkeeping 
68 | 2.15.3 438.416 and reporting 





Continuation of 
Benefits while the 
MCO/PIHP 
Appeal and the 
State Fair Hearing 
69 | 2.15.6(p) 4388.420 are Pending 





Effectuation of 
70 | 2.15(q,r) 438.424 reversed appeals 


























Total All Items 





This protocol was developed using the CMS MCO Compliance protocol worksheet and cross- 
matching the State of Missouri Eastern/Central Region contract and the State supplied 
Compliance Tool. 
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8 C Behavioral Health Concepts, Inc. 
® 4250 East Broadway, Suite 1055, Columbia, MO 65201 (573) 446-0405 


(573) 446-1816 (fax) 
(866) 463-6242 (toll-free) 
www. bhcinfo.com 


MCHP: 

Member Name: 

Case Manager Name: 
CM Service Type: 
Reviewer: 


Service 
Content: 


2011 External Quality Review — Case Review Tool 


After initial referral — 





> Member was contacted and Case Management was initiated. Yes__(if yes proceed to question #1). 
No__ 

>If No, was the member contacted within time frames? Yes No 

> Were required efforts made to contact the member and establish a relationship? Yes No | 

> Did member refuse services? Yes No 





> Reason given for not providing case management services: 


When a case is opened for services: 


Introduction to Case Management 
1. Is all identifying information available, including contact information? Y N 


2.Does narrative contain introductory information to members, such as: 


a. Explanation of Case Management services. Y N 

b The member’s right to accept/reject CM services. Y___—SsN 

C. Was obtaining member’s permission a problem? Y__N N/A 
d Third party disclosure circumstances were explained. Y N 
3. Is the reason for CM services provided? Y__—sN 


Comprehensive Assessment 
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4. Does the case record contain an assessment? Y N 
5. Was the assessment comprehensive and completed within required time frames? Y N 


The assessment for CM was within 30 days of enrolment for a new member; 
The assessment for CM was within 30 days of diagnosis for existing members. 


Comprehensive Care Planning 


6. Does this record contain care plans? Y N 

a. Is there evidence of member participation in care plan development? Y N 

b. Is there evidence that the care plan was coordinated and/or discussed with the member’s PCP? 
YN 


Type of Service Required 











7. Was the member part of a special program population? Y N 

a. Did the Case Manager follow MCHP protocols in serving this member? Y N 

8. Is this member pregnant? Y N 

a. If yes, was case management offered? Y N 

b. Was a risk assessment completed? Y N 

C. Is it included in the case record? Y N 

9 Is this a lead involved case? Y N 

a If yes, were case management services initiated within required time frames? Y N 

b. Did the initiation of services indicate which of the following categories the member is in? 
Y N 


i. 10 to 19 ug/dL within 1-3 days 
ii. 20 to 44 ug/dL within 1-2 days 
iii. 45 to 60 ug/dL within 24 hours 
iv. 70 ug/dL or greater — immediately 


10. Did the record indicate a diagnosis of: (check any that apply) 


Cancer 

Cardiacdisease_ | 

Chronic pain ___ 

HepatitisC 

HIV/AIDS 

Children with Special Healthcare Needs including Autism Spectrum Disorder____ 

Members with Special Healthcare Needs without services —__ 

(These may include, but not be limited to private duty nursing, home health, durable medical 
equipment/supplies, and/or a need for hospitalization or institutionalization.) 


Appropriate Provider and Service Referrals 


11. Were appropriate referrals made for necessary services that were not in place at the time of the 
assessment, or when recommended by the members’ physician/healthcare team? Y N N/A 
12. Were appropriate referrals made for community-based services? Y N N/A 
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a. Transportation services? Ys N/A 


Face to Face Contacts 


13. Is there evidence in the case record that face-to-face contacts occurred, as required? 
Y  N NA 
14. Who conducted face-to-face contacts? 


Progress Notes and Required Contacts 


15. Does this case record include progress notes as required? Y N 
16. Is there evidence that at least three (3) substantial contacts were made, directly with the member or 
their representative, prior to case closure? Y N 


PCP Involvement 





17. Do the case notes indicate if the PCP was informed that a case manager was working with the 
member? Y N 

a. Was the PCP informed when the case management record was closed? Y N Not 
Closed 

18. Was any history or additional information provided to or obtained from the PCP? Y N 


Case/Care Coordination 


19. Is there any evidence that the member was referred to Disease Management, if appropriate? 
YN N/A 

20. Is there evidence of care coordination in complex cases, asrequired?Y = = = N == NA 

21. Are behavioral health services discussed with the member? YN SNA 

22. When behavioral health services are deemed necessary is the PCP informed? Y_ = =N = NA ___- 
23. Is there evidence of care coordination with the behavioral healthCM?Y = =N NA 


Transition Plan and Case Closure 


24. If case closure has occurred, is there evidence that the member has achieved all stated care plan 
goals? Y N N/A 

25. Is there evidence that an appropriate transition of care was offered to the member, and followed at 
the time a case was closed? Y N N/A 

26. Do proper case closing criteria exist based on the type of case management received? 

YN N/A 


Additional Questions regarding this case or member situation that should be included in CM interviews: 
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